[CentOS] CentOS 7 grub.cfg missing on new install

Wed Dec 10 18:13:53 UTC 2014
Jeff Boyce <jboyce at meridianenv.com>

Greetings -

The short story is that got my new install completed with the partitioning I 
wanted and using software raid, but after a reboot I ended up with a grub 
prompt, and do not appear to have a grub.cfg file.  So here is a little 
history of how I got here, because I know in order for anyone to help me 
they would subsequently ask for this information.  So this post is a little 
long, but consider it complete.

Brand new Dell system with two 3GB drives in this system with RAID1 LVM 
taking all the space outside the boot partitions.  I initially created the 
sda[1,2] and sdb[1,2] partitions via GParted leaving the remaining space 
unpartitioned.  A gpt partition table was put on both drives.  During 
installation Anaconda recognized everything properly which resulted in the 
following partition summary:

sda1 /boot/efi 500 MB EFI System Partition
sda2 /boot 500 MB xfs
vg_jab-hostroot / 8 GB LVM xfs RAID1
vg_jab-hostvar /var 4 GB LVM xfs RAID1
vg_jab-hostswap /swap 2 GB LVM swap RAID1

The installer also recognized and listed these unknown partitions that were 
untouched during installation.
sdb1 vfat 500 MB standard partition
sdb2 vfat 500 MB standard partition

Installation proceeded successfully, and after the initial reboot of the 
system I used mdadm commands to watch the raid complete building before 
doing anything else (I know, not necessary but I am doing other things and 
had the time to let it complete).  I rebooted the system and got a terminal 
prompt as expected (no GUI installed).  At this point I needed to copy my 
/boot/efi and /boot partitions from sda[1,2] to sdb[1,2] so that the system 
would boot from either drive, so I issued the following sgdisk commands:

root#  sgdisk -R /dev/sdb1 /dev/sda1
root#  sgdisk -R /dev/sdb2 /dev/sda2
root#  sgdisk -G /dev/sdb1
root#  sgdisk -G /dev/sdb2

Results of the first command above:
   Found invalid GPT and valid MBR; converting MBR to GPT format.
   Warning the kernel is still using the old partition table.  The new table 
will be used at the next reboot.  The operation has completed successfully.
   The same note (from the Warning on) was repeated for the other three 
commands.

I then installed GRUB2 on /dev/sdb1 using the following command:
root#  grub2-install /dev/sdb1
    Results:  Installing for x86_64-efi platform.  Installation finished. No 
error reported.

I rebooted the system now, only to be confronted with a GRUB prompt. 
Thinking that this is a good opportunity to for me to learn to rescue a 
system since I am going to need to understand how to recover from a disk or 
raid failure, I started researching and reading.  It takes a little bit of 
work to understand what information is valuable when a lot of it refers to 
GRUB (not GRUB2) and doesn't make reference to UEFI booting and partitions. 
I found this Ubuntu wiki as a pretty good source
https://help.ubuntu.com/community/Grub2/Troubleshooting#Search_.26_Set

Below is the current information of my system as seen by grub;

grub#  set    (the important grub2 variables are:)
                prefix = (hd1, gpt2)/grub2
                root = hd1, gpt2

grub#  ls -lha
                Device proc:  filesystem type procfs
                Device hd0:  no known filesystem detected
                Device hd1:  no known filesystem detected
                    Partition hd1, gpt3:  no known filesystem detected
                    Partition hd1, gpt2:  filesystem xfs
                    Partition hd1, gpt1:  filesystem fat
                Device hd2:  no known filesystem detected
                    Partition hd2, gpt3:  no known filesystem detected
                    Partition hd2, gpt2:  no known filesystem detected
                    Partition hd2, gpt1:  no known filesystem detected

grub#  ls (hd1, gpt2) -l /
                /efi
                /grub
                /grub2
                vmlinuz -3.10.0-123.el7.x86_64
                initramfs-3.10.0-123.el7.x86_64.img
                ... plus some other files

Looking through the directories, I see that there is no grub.cfg file. 
Other than grub not recognizing the filesystem on hd2, the directories on 
hd2, gpt[1,2] seem to be identical to hd1, gpt[1,2] as I would assume based 
on the sgdisk commands I gave to copy them.  My initial thinking is that I 
need to re-run grub2-install on hd1 (sda), but I need a running system to do 
that.  So following the guidance I had I issued the following commands in 
grub to boot the system.

grub#  linux /vmlinuz -3.10.0-123.el7.x86_64 root=/dev/sda2 ro
grub#  initrd /initramfs-3.10.0-123.el7.x86_64.img
grub#  boot

Unfortunately the system hung on booting, with the following information in 
the "journalctl" file:
#  journalctl
Not switching root: /sysroot does not seem to be an OS tree. 
/etc/os-release is missing.
Initrd-switch-root.service: main process exited, code=exited, 
status=1/FAILURE
Failed to start Switch Root.
. . . . .
Triggering OnFailure= dependencies of initrd-switch-root.service.
Starting Emergency Shell. . .
Failed to issue method call: Invalid argument

Now I am not sure that I want to get misdirected to what the problem is with 
this boot, if I can boot from a CD in linux rescue mode and do the grub 
install, then be back to a booting system.  So lets ignore the boot error if 
we can.  So I boot from a CD in rescue mode, and it is only able to 
automatically mount sd3 under /mnt/sysimage (the LVM RAID1 containing mounts 
for / and /var).  I am able to manually mount sda1 and sda2, but am not sure 
at what level in the filesystem to mount them (i.e., at /mnt/sda1 or at 
mnt/sysimage/sda1) in order to properly run grub2-install.

So that is where I am at now.  I would like to know how to repair the 
system, rather than starting over on a new install.  Can someone enlighten 
me on what I need to do from here.  Also if someone can speculate on why my 
grub.cfg is missing in the first place I would be interested.

Also, please cc me directly on any responses, as I am only subscribed to the 
daily digest.  Thanks.

Jeff Boyce
Meridian Environmental
www.meridianenv.com