Hello everyone.
I'm trying to install CentOS 8 with root and swap partitions on software raid. The plan is: - create md0 raid level 1 with 2 hard drives: /dev/sda and /dev/sdb, using Linux Rscue CD, - install CentOS 8 with Virtual Box on my laptop, - rsync CentOS 8 root partition on /dev/md0p1, - chroot in CentOS 8 root partition, - configure /etc/mdadm.conf, grub.cfg, initramfs, install bootloader on both sda and sdb drives.
I think I can do first four of the above, but my CentOS installation acts strange after rebooting the server. It recognizes the raid, but boots randomly with root on /dev/sda1 (and recognizes raid with /dev/sdb disk), or with root on /dev/sdb1 (and recognizes raid with /dev/sda disk). When booting from Linux Rescue CD, the raid with two disk is recognized.
Creating md0 wth Linux Rescue CD.
# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT loop0 7:0 0 819.5M 1 loop /run/archiso/sfs/airootfs sda 8:0 0 232.9G 0 disk sdb 8:16 0 232.9G 0 disk sr0 11:0 1 871M 0 rom
# mdadm --create /dev/md0 --level=1 --raid-devices=2 --metadata=0.9 /dev/sda /dev/sdb mdadm: array /dev/md0 started.
# mdadm --detail --scan --verbose ARRAY /dev/md0 level=raid1 num-devices=2 metadata=0.90 UUID=fef2f68b: 1a462c80:c44c77eb:7ee19756 devices=/dev/sda,/dev/sdb
# cat /proc/mdstat Personalities : [raid1] md0 : active raid1 sdb[1] sda[0] 244198464 blocks [2/2] [UU] [>....................] resync = 1.9% (4682944/244198464) finish=52.2min speed=76356K/sec bitmap: 2/2 pages [8KB], 65536KB chunk
unused devices: <none>
(1 hour later)
# cat /proc/mdstat Personalities : [raid1] md0 : active raid1 sdb[1] sda[0] 244198464 blocks [2/2] [UU] bitmap: 0/2 pages [0KB], 65536KB chunk
unused devices: <none>
# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT loop0 7:0 0 819.5M 1 loop /run/archiso/sfs/airootfs sda 8:0 0 232.9G 0 disk └─md0 9:0 0 232.9G 0 raid1 sdb 8:16 0 232.9G 0 disk └─md0 9:0 0 232.9G 0 raid1 sr0 11:0 1 871M 0 rom
---------------------------------
So far so good. Now partitions:
# fdisk /dev/md0
Welcome to fdisk (util-linux 2.33.1). Changes will remain in memory only, until you decide to write them. Be careful before using the write command.
Device does not contain a recognized partition table. Created a new DOS disklabel with disk identifier 0x64998720.
Command (m for help): o Created a new DOS disklabel with disk identifier 0xfe784f57.
Command (m for help): n Partition type p primary (0 primary, 0 extended, 4 free) e extended (container for logical partitions) Select (default p): p Partition number (1-4, default 1): First sector (2048-488396927, default 2048): Last sector, +/-sectors or +/-size{K,M,G,T,P} (2048-488396927, default 488396927): +220G
Created a new partition 1 of type 'Linux' and of size 220 GiB.
Command (m for help): n Partition type p primary (1 primary, 0 extended, 3 free) e extended (container for logical partitions) Select (default p): p Partition number (2-4, default 2): First sector (461375488-488396927, default 461375488): Last sector, +/-sectors or +/-size{K,M,G,T,P} (461375488-488396927, default 488396927): +4G
Created a new partition 2 of type 'Linux' and of size 4 GiB.
Command (m for help): w The partition table has been altered. Calling ioctl() to re-read partition table. Syncing disks.
# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT loop0 7:0 0 819.5M 1 loop /run/archiso/sfs/airootfs sda 8:0 0 232.9G 0 disk └─md0 9:0 0 232.9G 0 raid1 ├─md0p1 259:0 0 220G 0 part └─md0p2 259:1 0 4G 0 part sdb 8:16 0 232.9G 0 disk └─md0 9:0 0 232.9G 0 raid1 ├─md0p1 259:0 0 220G 0 part └─md0p2 259:1 0 4G 0 part sr0 11:0 1 871M 0 rom
# mkfs -t ext4 /dev/md0p1 mke2fs 1.44.5 (15-Dec-2018) Creating filesystem with 57671680 4k blocks and 14417920 inodes Filesystem UUID: 511d0d07-4d55-4b6c-939e-d23ae2da1ce0 Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 4096000, 7962624, 11239424, 20480000, 23887872
Allocating group tables: done Writing inode tables: done Creating journal (262144 blocks): done Writing superblocks and filesystem accounting information: done
# mkswap /dev/md0p2 Setting up swapspace version 1, size = 4 GiB (4294963200 bytes) no label, UUID=5667d007-0d3e-4b54-8a72-6ade52609adc
--------------------------------------
Mount would be root partition:
# mkdir /mnt/centos8 # mount -t ext4 /dev/md0p1 /mnt/centos8/
--------------------------------------
OK, I have what I need. Now log into virtualised CentOS 8 in Virtual Box.
# rpm -q rsync rsync-3.1.3-7.el8.x86_64 # rpm -q mdadm mdadm-4.1-13.el8.x86_64
# rsync -vaxAXHSz --delete / 192.168.111.167:/mnt/centos8/ The authenticity of host '192.168.111.167 (192.168.111.167)' can't be established. ECDSA key fingerprint is SHA256:HukswAQlarthYqbpVHIHm80JaWHKX12AxgOn8UuKQxk. Are you sure you want to continue connecting (yes/no/[fingerprint])? yes Warning: Permanently added '192.168.111.167' (ECDSA) to the list of known hosts. root@192.168.111.167's password: sending incremental file list ./ bin -> usr/bin lib -> usr/lib lib64 -> usr/lib64 sbin -> usr/sbin [lots and lots of files...] var/yp/
sent 694,980,592 bytes received 745,802 bytes 1,795,422.95 bytes/sec total size is 1,558,934,091 speedup is 2.24
--------------------------------------
Chrooting time:
# mount --types proc /proc /mnt/centos8/proc # mount --rbind /sys /mnt/centos8/sys # mount --make-rslave /mnt/centos8/sys # mount --rbind /dev /mnt/centos8/dev # mount --make-rslave /mnt/centos8/dev # chroot /mnt/centos8 /usr/bin/bash # source /etc/profile
Configure raid:
# mdadm --detail --scan --verbose ARRAY /dev/md0 level=raid1 num-devices=2 metadata=0.90 UUID=fef2f68b: 1a462c80:c44c77eb:7ee19756 devices=/dev/sda,/dev/sdb # mdadm --detail --scan --verbose > /etc/mdadm.conf
Edit /etc/fstab:
# cat /etc/fstab
# # /etc/fstab # Created by anaconda on Sun Nov 8 10:05:31 2020 # # Accessible filesystems, by reference, are maintained under # '/dev/disk/'. See man pages fstab(5), findfs(8), mount(8) and/or # blkid(8) for more info. # # After editing this file, run 'systemctl daemon-reload' to update # systemd units generated from this file. # #UUID=36c5a887-79c3-4b6a-b433-0e4d7574b1e4 / #ext4 defaults 1 1 /dev/md0p1 / ext4 defaults 1 1
Configure and install bootloader:
# cat /etc/default/grub GRUB_TIMEOUT=8 GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)" GRUB_DEFAULT=saved GRUB_DISABLE_SUBMENU=true GRUB_TERMINAL_OUTPUT="console" GRUB_CMDLINE_LINUX="rhgb quiet" GRUB_DISABLE_RECOVERY="true" GRUB_ENABLE_BLSCFG=true
# grub2-mkconfig -o /boot/grub2/grub.cfg Generating grub configuration file ... done
# grub2-install /dev/sda Installing for i386-pc platform. Installation finished. No error reported. # grub2-install /dev/sdb Installing for i386-pc platform. Installation finished. No error reported.
Configure initramfs:
# dracut -f /boot/initramfs-4.18.0-193.28.1.el8_2.x86_64.img # 4.18.0-193.28.1.el8_2.x86_64
--------------------------------
Reboot and login to my new CentOS 8 installation:
# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 232.9G 0 disk sdb 8:16 0 232.9G 0 disk ├─sdb1 8:17 0 220G 0 part / └─sdb2 8:18 0 4G 0 part sr0 11:0 1 871M 0 rom
# mdadm --detail --scan --verbose INACTIVE-ARRAY /dev/md127 num-devices=1 metadata=0.90 UUID=fef2f68b: 1a462c80:c44c77eb:7ee19756 devices=/dev/sda
# cat /proc/mdstat Personalities : md127 : inactive sda[0](S) 244198464 blocks
unused devices: <none>
I can live with md127, because it happens constantly on multiple distributions, but why root is on /dev/sdb1 (and sometimes it's /dev/sda1)? I'm running out of spells now. :) Do anyone can suggest what else I forgot to do?
thanks