I plan to upgrade an existing C7 computer which currently has one 256 GB SSD to use mdadmin software RAID1 after adding two 4 TB M2. SSDs, the rest of the system remaining the same. The system also has one additional internal and one external harddisk but these should not be touched. The system will continue to run C7.
If I remember correctly, the existing SSD does not use a M2. slot so they should be available for the new 4GB SSDs while the old SSD remains in place. If I read the output from gparted correctly, this existing SSD is partitioned as follows:
- EFI system partition 260 MB and mounted as /boot/efi (formatted as FAT32).
- boot partition 1 GB and mounted as /boot (formatted as xfs).
- /root, swap and /home seem to coexist on a LUKS encrypted partitition using LVM2 PV.
My current plan is to:
- Use clonezilla to make individual backups of all partitions above.
- Install the two M2. SSDs.
- Use an external disk tool to partition the two new M2. SSDs as follows:
-- Create a RAID1 partition for /boot - same size as current, ie 260 MB
-- Create a RAID1 partition for /boot/efi - same size as current, ie 1 GB
-- Create a RAID1 partition for LVM2 and LUKS - the rest of the 4 TB SSD
Questions:
- I do not see any benefit to breaking up the LVM2/LUKS partition containing /root, /swap and /home into more than one RAID1 partition or am I wrong? If the SSD fails, the entire SSD would fail and break the system, hence I might as well keep it as one single RAID1 partition, or?
- Is the next step after the RAID1 partitioning above then to do a minimal install of C7 followed by using clonezilla to restoring the LVM2/LUKS partition??
- Any advice on using clonezilla? Or the external partitioning tool?
- Finally, since these new SSDs are huge, perhaps I should take the opportunity to increase the space for both /root and /swap?
- /root is 50 GB - should I increase it to eg 100 GB?
- The system currently has 32 GB of memory but I will likely upgrade it to 64 GB (or even 128 GB), perhaps I should at this time already increase the /swap space to 64 GB/128 GB?
Looking forward to advice and suggestions!
I plan to upgrade an existing C7 computer which currently has one 256 GB SSD to use mdadmin software RAID1 after adding two 4 TB M2. SSDs, the rest of the system remaining the same. The system also has one additional internal and one external harddisk but these should not be touched. The system will continue to run C7.
If I remember correctly, the existing SSD does not use a M2. slot so they should be available for the new 4GB SSDs while the old SSD remains in place. If I read the output from gparted correctly, this existing SSD is partitioned as follows:
- EFI system partition 260 MB and mounted as /boot/efi (formatted as
FAT32).
boot partition 1 GB and mounted as /boot (formatted as xfs).
/root, swap and /home seem to coexist on a LUKS encrypted partitition
using LVM2 PV.
My current plan is to:
Use clonezilla to make individual backups of all partitions above.
Install the two M2. SSDs.
Use an external disk tool to partition the two new M2. SSDs as follows:
-- Create a RAID1 partition for /boot - same size as current, ie 260 MB
-- Create a RAID1 partition for /boot/efi - same size as current, ie 1 GB
-- Create a RAID1 partition for LVM2 and LUKS - the rest of the 4 TB SSD
Questions:
- I do not see any benefit to breaking up the LVM2/LUKS partition
containing /root, /swap and /home into more than one RAID1 partition or am I wrong? If the SSD fails, the entire SSD would fail and break the system, hence I might as well keep it as one single RAID1 partition, or?
What I usually do is this: "cut" the large disk into several pieces of equal size and create individual RAID1 arrays. Then add them as LVM PVs to one large VG. The advantage is that with one error on one disk, you wont lose redundancy on the whole RAID mirror but only on a partial segment. You can even lose another segment with an error on the other disk and still have redundancy if the error is in another part.
That said, it's a bit more work to setup but has helped me several times in the decades ago.
- Is the next step after the RAID1 partitioning above then to do a minimal
install of C7 followed by using clonezilla to restoring the LVM2/LUKS partition??
- Any advice on using clonezilla? Or the external partitioning tool?
- Finally, since these new SSDs are huge, perhaps I should take the opportunity to increase the space for both /root and /swap?
/root is 50 GB - should I increase it to eg 100 GB?
The system currently has 32 GB of memory but I will likely upgrade it to
64 GB (or even 128 GB), perhaps I should at this time already increase the /swap space to 64 GB/128 GB?
I'm also interested here to learn what others are doing in higher memory situations. I have some systems with half a TB memory and never configured more than 16GB of swap. I has usually worked well and when a system started to use swap heavily, there was something really wrong in an application and had to be fixed there. Additionally we've tuned the kernel VM settings so that it didn't want to swap too much. Because swapping was always slow anyway even on fast U.2 NVME SSD storage.
Regards, Simon
On 1/11/23 02:09, Simon Matter wrote:
I plan to upgrade an existing C7 computer which currently has one 256 GB SSD to use mdadmin software RAID1 after adding two 4 TB M2. SSDs, the rest of the system remaining the same. The system also has one additional internal and one external harddisk but these should not be touched. The system will continue to run C7.
.... trimming
- I do not see any benefit to breaking up the LVM2/LUKS partition
containing /root, /swap and /home into more than one RAID1 partition or am I wrong? If the SSD fails, the entire SSD would fail and break the system, hence I might as well keep it as one single RAID1 partition, or?
What I usually do is this: "cut" the large disk into several pieces of equal size and create individual RAID1 arrays. Then add them as LVM PVs to one large VG. The advantage is that with one error on one disk, you wont lose redundancy on the whole RAID mirror but only on a partial segment. You can even lose another segment with an error on the other disk and still have redundancy if the error is in another part.
That said, it's a bit more work to setup but has helped me several times in the decades ago.
Ah, now I begin to get it. Separate partitions RAIDed.
- Is the next step after the RAID1 partitioning above then to do a minimal
install of C7 followed by using clonezilla to restoring the LVM2/LUKS partition??
- Any advice on using clonezilla? Or the external partitioning tool?
- Finally, since these new SSDs are huge, perhaps I should take the opportunity to increase the space for both /root and /swap?
/root is 50 GB - should I increase it to eg 100 GB?
The system currently has 32 GB of memory but I will likely upgrade it to
64 GB (or even 128 GB), perhaps I should at this time already increase the /swap space to 64 GB/128 GB?
I'm also interested here to learn what others are doing in higher memory situations. I have some systems with half a TB memory and never configured more than 16GB of swap. I has usually worked well and when a system started to use swap heavily, there was something really wrong in an application and had to be fixed there. Additionally we've tuned the kernel VM settings so that it didn't want to swap too much. Because swapping was always slow anyway even on fast U.2 NVME SSD storage.
Perhaps you have not dealt with Firefox? :)
On my Fedora 35 notebook, it slowly gobbles memory and I have to quit it after some number of days and restart.
Now I only have 16GB of memory, 16GB physical swap, and 8GB zram swap.
Building a F37 system now and see how that works, I doubt there is any improved behavior with Firefox.
Regards, Simon
CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
On 1/11/23 02:09, Simon Matter wrote:
I plan to upgrade an existing C7 computer which currently has one 256 GB SSD to use mdadmin software RAID1 after adding two 4 TB M2. SSDs, the rest of the system remaining the same. The system also has one additional internal and one external harddisk but these should not be touched. The system will continue to run C7.
.... trimming
- I do not see any benefit to breaking up the LVM2/LUKS partition
containing /root, /swap and /home into more than one RAID1 partition or am I wrong? If the SSD fails, the entire SSD would fail and break the system, hence I might as well keep it as one single RAID1 partition, or?
What I usually do is this: "cut" the large disk into several pieces of equal size and create individual RAID1 arrays. Then add them as LVM PVs to one large VG. The advantage is that with one error on one disk, you wont lose redundancy on the whole RAID mirror but only on a partial segment. You can even lose another segment with an error on the other disk and still have redundancy if the error is in another part.
That said, it's a bit more work to setup but has helped me several times in the decades ago.
Ah, now I begin to get it. Separate partitions RAIDed.
Yes, exactly, sorry for not being clear enough. The structure of disk 0 looks like this:
$ lsblk -o NAME,FSTYPE,SIZE,MOUNTPOINT /dev/nvme0n1 NAME FSTYPE SIZE MOUNTPOINT nvme0n1 745.2G ├─nvme0n1p1 vfat 200M /boot/efi ├─nvme0n1p2 linux_raid_member 1G │ └─md0 xfs 1023M /boot ├─nvme0n1p3 linux_raid_member 186G │ └─md1 LVM2_member 185.9G │ ├─main-root xfs 30G / │ ├─main-swap swap 16G [SWAP] │ ├─main-tmp xfs 10G /tmp │ ├─main-var xfs 200G /var │ └─main-imap xfs 1.2T /var/spool/imap ├─nvme0n1p4 linux_raid_member 186G │ └─md2 LVM2_member 185.9G │ └─main-imap xfs 1.2T /var/spool/imap ├─nvme0n1p5 linux_raid_member 186G │ └─md3 LVM2_member 185.9G │ └─main-imap xfs 1.2T /var/spool/imap ├─nvme0n1p6 linux_raid_member 186G │ └─md4 LVM2_member 185.9G │ ├─main-var xfs 200G /var │ └─main-imap xfs 1.2T /var/spool/imap └─nvme0n1p7 15.8M
Regards, Simon
On 01/11/2023 08:34 AM, Robert Moskowitz wrote:
Perhaps you have not dealt with Firefox? :)
On my Fedora 35 notebook, it slowly gobbles memory and I have to quit it after some number of days and restart.
Now I only have 16GB of memory, 16GB physical swap, and 8GB zram swap.
Building a F37 system now and see how that works, I doubt there is any improved behavior with Firefox.
For me, Chrome is the biggest memory gobbler... I think I saw that the latest release may release memory when tabs are not loaded which sounds like a big step in the right direction.
On 01/11/2023 02:09 AM, Simon Matter wrote:
What I usually do is this: "cut" the large disk into several pieces of equal size and create individual RAID1 arrays. Then add them as LVM PVs to one large VG. The advantage is that with one error on one disk, you wont lose redundancy on the whole RAID mirror but only on a partial segment. You can even lose another segment with an error on the other disk and still have redundancy if the error is in another part.
That said, it's a bit more work to setup but has helped me several times in the decades ago.
But is your strategy of dividing the large disk into individual RAID1 arrays also applicable to SSDs? I have heard, perhaps incorrectly, that once a SSD fails, the entire SSD becomes unusable which would suggest that dividing it into multiple RAID1 arrays would not be useful?
On 01/11/2023 01:33 PM, H wrote:
On 01/11/2023 02:09 AM, Simon Matter wrote:
What I usually do is this: "cut" the large disk into several pieces of equal size and create individual RAID1 arrays. Then add them as LVM PVs to one large VG. The advantage is that with one error on one disk, you wont lose redundancy on the whole RAID mirror but only on a partial segment. You can even lose another segment with an error on the other disk and still have redundancy if the error is in another part.
That said, it's a bit more work to setup but has helped me several times in the decades ago.
But is your strategy of dividing the large disk into individual RAID1 arrays also applicable to SSDs? I have heard, perhaps incorrectly, that once a SSD fails, the entire SSD becomes unusable which would suggest that dividing it into multiple RAID1 arrays would not be useful?
Follow-up question: Is my proposed strategy below correct:
- Make a copy of all existing directories and files on the current disk using clonezilla.
- Install the new M.2 SSDs.
- Partitioning the new SSDs for RAID1 using an external tool.
- Doing a minimal installation of C7 and mdraid.
- If choosing three RAID partitions, one for /boot, one for /boot/efi and the third one for the rest, do I go with the default mdraid version, ie 1.2 I believe?
- Copying the backup above with contents of the the existing disks, ie not just /root and /home but all other directories and files to the new disks from the clonezilla backup. Note that the new disks will be larger.
- Change the boot sequence in the BIOS and reboot.
Thanks.
On 01/11/2023 01:33 PM, H wrote:
On 01/11/2023 02:09 AM, Simon Matter wrote:
What I usually do is this: "cut" the large disk into several pieces of equal size and create individual RAID1 arrays. Then add them as LVM PVs to one large VG. The advantage is that with one error on one disk, you wont lose redundancy on the whole RAID mirror but only on a partial segment. You can even lose another segment with an error on the other disk and still have redundancy if the error is in another part.
That said, it's a bit more work to setup but has helped me several times in the decades ago.
But is your strategy of dividing the large disk into individual RAID1 arrays also applicable to SSDs? I have heard, perhaps incorrectly, that once a SSD fails, the entire SSD becomes unusable which would suggest that dividing it into multiple RAID1 arrays would not be useful?
Follow-up question: Is my proposed strategy below correct:
- Make a copy of all existing directories and files on the current disk
using clonezilla.
Install the new M.2 SSDs.
Partitioning the new SSDs for RAID1 using an external tool.
Doing a minimal installation of C7 and mdraid.
If choosing three RAID partitions, one for /boot, one for /boot/efi and
the third one for the rest, do I go with the default mdraid version, ie 1.2 I believe?
I think at least /boot/efi must be on an mdraid version which has its metadata at the end of the partition, I'm not sure about /boot. That said, I think the installer should take care here but I'm not sure it already does on C7.
- Copying the backup above with contents of the the existing disks, ie not
just /root and /home but all other directories and files to the new disks from the clonezilla backup. Note that the new disks will be larger.
I can't comment on clonezilla as I've never used it. Tar and rsync are my friends when doing such things. I think you may have to take special care for boot related stuff like things in /boot and boot/efi. The other thing to care is for hardware related stuff like UUIDs generated in /etc/udev. The whole undertaking is not trivial.
- Change the boot sequence in the BIOS and reboot.
That's EFI, yes? I still fell like a greenhorn with EFI :)
Regards, Simon
On January 12, 2023 1:56:36 AM EST, Simon Matter simon.matter@invoca.ch wrote:
On 01/11/2023 01:33 PM, H wrote:
On 01/11/2023 02:09 AM, Simon Matter wrote:
What I usually do is this: "cut" the large disk into several pieces
of
equal size and create individual RAID1 arrays. Then add them as LVM
PVs
to one large VG. The advantage is that with one error on one disk, you wont lose redundancy on the whole RAID mirror but only on a partial
segment.
You can even lose another segment with an error on the other disk
and
still have redundancy if the error is in another part.
That said, it's a bit more work to setup but has helped me several times in the decades ago.
But is your strategy of dividing the large disk into individual
RAID1
arrays also applicable to SSDs? I have heard, perhaps incorrectly,
that
once a SSD fails, the entire SSD becomes unusable which would
suggest
that dividing it into multiple RAID1 arrays would not be useful?
Follow-up question: Is my proposed strategy below correct:
- Make a copy of all existing directories and files on the current
disk
using clonezilla.
Install the new M.2 SSDs.
Partitioning the new SSDs for RAID1 using an external tool.
Doing a minimal installation of C7 and mdraid.
If choosing three RAID partitions, one for /boot, one for /boot/efi
and
the third one for the rest, do I go with the default mdraid version,
ie
1.2 I believe?
I think at least /boot/efi must be on an mdraid version which has its metadata at the end of the partition, I'm not sure about /boot. That said, I think the installer should take care here but I'm not sure it already does on C7.
- Copying the backup above with contents of the the existing disks,
ie not
just /root and /home but all other directories and files to the new
disks
from the clonezilla backup. Note that the new disks will be larger.
I can't comment on clonezilla as I've never used it. Tar and rsync are my friends when doing such things. I think you may have to take special care for boot related stuff like things in /boot and boot/efi. The other thing to care is for hardware related stuff like UUIDs generated in /etc/udev. The whole undertaking is not trivial.
- Change the boot sequence in the BIOS and reboot.
That's EFI, yes? I still fell like a greenhorn with EFI :)
Regards, Simon
CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Upon further thinking, I am wondering if the process below would work? I have now have two working disk setups in the same computer and depending on the order of disks in the BIOS I can boot any of the two setups.
My existing setup uses one disk and no RAID (obviously), LUKS and LVM for everything but /boot and /boot/efi, total of three partitions. The OS is Centos 7 and I have made a complete backup to an external harddisk using rsync ("BACKUP1").
The new one uses two disks, RAID1, LUKS and LVM for everything but /boot and /boot/efi, total of four partitions (swap has its own partition - not sure why I made it that way). A minimal installation of Centos 7 was made to this setup and is working. In other words, UUIDs of disks, partitions and LUKS are already configured and working.
So, I am now thinking the following might work:
- Make a rsync backup of the new disks to the external harddisk ("BACKUP2").
- Delete all files from the new disks except from /boot and /boot/efi.
- Copy all files from all partitions except /boot and /boot/efi from BACKUP1 to the new disks. In other words, everything except /boot and /boot/efi will now be overwritten.
- I would expect this system not to boot since both /etc/fstab and /etc/crypttab on the new disks contain the UUIDs from the old system.
- Copy just /etc/fstab and /etc/crypttab from BACKUP2 to the new disks. This should update the new disks with the previously created UUIDs from when doing the minimal installation of CentOS 7.
What do you think?
Follow-up question: Is my proposed strategy below correct:
- Make a copy of all existing directories and files on the current disk using clonezilla.
- Install the new M.2 SSDs.
- Partitioning the new SSDs for RAID1 using an external tool.
- Doing a minimal installation of C7 and mdraid.
- If choosing three RAID partitions, one for /boot, one for /boot/efi and the third one for the rest, do I go with the default mdraid version, ie 1.2 I believe?
- Copying the backup above with contents of the the existing disks, ie not just /root and /home but all other directories and files to the new disks from the clonezilla backup. Note that the new disks will be larger.
- Change the boot sequence in the BIOS and reboot.
I don't know if this is still a problem, but in the past, CentOS was only writing the boot-loader on one of the RAID-disks. If this one would fail, the OS would not boot. So you had to make sure to copy the boot-loader onto all RAID-members.
Michael
On January 12, 2023 2:43:26 AM EST, Michael Schumacher michael_ml@gmx.de wrote:
Follow-up question: Is my proposed strategy below correct:
- Make a copy of all existing directories and files on the current
disk using clonezilla.
- Install the new M.2 SSDs.
- Partitioning the new SSDs for RAID1 using an external tool.
- Doing a minimal installation of C7 and mdraid.
- If choosing three RAID partitions, one for /boot, one for /boot/efi
and the third one for the rest, do I go with the default mdraid version, ie 1.2 I believe?
- Copying the backup above with contents of the the existing disks,
ie not just /root and /home but all other directories and files to the new disks from the clonezilla backup. Note that the new disks will be larger.
- Change the boot sequence in the BIOS and reboot.
I don't know if this is still a problem, but in the past, CentOS was only writing the boot-loader on one of the RAID-disks. If this one would fail, the OS would not boot. So you had to make sure to copy the boot-loader onto all RAID-members.
Michael
Ok, will check. Thank you for raising this issue.
On 01/11/2023 02:09 AM, Simon Matter wrote:
What I usually do is this: "cut" the large disk into several pieces of equal size and create individual RAID1 arrays. Then add them as LVM PVs to one large VG. The advantage is that with one error on one disk, you wont lose redundancy on the whole RAID mirror but only on a partial segment. You can even lose another segment with an error on the other disk and still have redundancy if the error is in another part.
That said, it's a bit more work to setup but has helped me several times in the decades ago.
But is your strategy of dividing the large disk into individual RAID1 arrays also applicable to SSDs? I have heard, perhaps incorrectly, that once a SSD fails, the entire SSD becomes unusable which would suggest that dividing it into multiple RAID1 arrays would not be useful?
What you heard seems extremely oversimplified to me. A HD can fail in different manners and so can SSDs. Anyway, the splitting of large disks has additional advantages. Think of what happens in case of a failure (power loss, kernel crash...). With the disk as one large chunk, the whole disk has to be resynced on restart while with smaller segments only those which are marked as dirty have to be resynced. This can make a bit difference.
Regards, Simon
Hallo Simon,
Anyway, the splitting of large disks has additional advantages. Think of what happens in case of a failure (power loss, kernel crash...). With the disk as one large chunk, the whole disk has to be resynced on restart while with smaller segments only those which are marked as dirty have to be resynced. This can make a bit difference.
I am not sure if this is true. If a underlying disk fails, it will mark all partitions on that disk as dirty, so you will have to resync them all after replacing or readding the disk into the array.
Michael
Hallo Simon,
Anyway, the splitting of large disks has additional advantages. Think of what happens in case of a failure (power loss, kernel crash...). With the disk as one large chunk, the whole disk has to be resynced on restart while with smaller segments only those which are marked as dirty have to be resynced. This can make a bit difference.
I am not sure if this is true. If a underlying disk fails, it will mark all partitions on that disk as dirty, so you will have to resync them all after replacing or readding the disk into the array.
No, I'm not talking about a complete disk failure, my example wasn't a failure at all but a server problem like power loss, kernel crash and such things. In this case only the segments which were not in sync at the time of the crash will be resynced on restart, not the whole disk.
The same is, if a read error happens on one disk, only the partial segment will lose redundancy and not the whole contents of the disk.
That's a huge improvement especially on very large disks.
Simon
Il 2023-01-12 09:00 Simon Matter ha scritto:
That's a huge improvement especially on very large disks.
Hi, not-ancient versions of Linux MD RAID cope with these issues via two different means: - a write bitmap to track dirty disk regions; - an embedded bad list sector to remap such sectors when appropriate (and without failing the entire array, if possible).
Still, splitting a disk into multiple slices has the specific advantage of using different RAID levels on different dataset (which can be very valuable in some cases).
Regards.
On 01/12/2023 03:00 AM, Simon Matter wrote:
Hallo Simon,
Anyway, the splitting of large disks has additional advantages. Think of what happens in case of a failure (power loss, kernel crash...). With the disk as one large chunk, the whole disk has to be resynced on restart while with smaller segments only those which are marked as dirty have to be resynced. This can make a bit difference.
I am not sure if this is true. If a underlying disk fails, it will mark all partitions on that disk as dirty, so you will have to resync them all after replacing or readding the disk into the array.
No, I'm not talking about a complete disk failure, my example wasn't a failure at all but a server problem like power loss, kernel crash and such things. In this case only the segments which were not in sync at the time of the crash will be resynced on restart, not the whole disk.
The same is, if a read error happens on one disk, only the partial segment will lose redundancy and not the whole contents of the disk.
That's a huge improvement especially on very large disks.
Simon
CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
I have not seen anyone comment on my plan to after partitioning the new SSDs that I have to do a new minimal install of C7 and then copy the old disk partitions - with the exceptions of /boot and /boot/efi - over the newly made installation?
Am I correct in that is needed since the old installation was not using RAID and and the new one does? Both of course are using C7.
-
Gionatan Danti -
H -
Michael Schumacher -
Robert Moskowitz -
Simon Matter