We have a working PXE setup. I've tried to adapt it to UEFI as per RHEL6 manual, but the client won't boot.
Strangely, tcpdump shows that the client tries to download all files via tftp from the dhcp server rather than the tftp server (they are different). next-server is pointing to the tftp server.
Any clues?
Once upon a time, isdtor isdtor@gmail.com said:
We have a working PXE setup. I've tried to adapt it to UEFI as per RHEL6 manual, but the client won't boot.
I have BIOS+UEFI PXE boot set up, although it took some doing. I still use SYSLINUX for BIOS, but GRUB2 plus Red Hat's secure boot shim for UEFI. I never could get SYSLINUX's UEFI support working very well or reliably, and it didn't support secure boot at all last I looked.
I also feed most of the content out via HTTP (much faster than TFTP). That includes generating menus on the fly from perl CGIs (I'm old-fashioned that way :) ).
I'm using dnsmasq as my DHCP server, so I have this in its config (where pxesrv.cmadams.net has IP 10.10.10.2):
******************** # EFI boot will set ARCH option (93) dhcp-match=set:efi64,option:client-arch,7 tag-if=set:bios,tag:!efi32,tag:!efi64
# PXE boot dhcp-boot=tag:efi64,shimx64.efi,pxesrv.cmadams.net,10.10.10.2 dhcp-boot=tag:bios,bios/lpxelinux.0,pxesrv.cmadams.net,10.10.10.2 dhcp-option-force=lan,209,pxelinux.conf ********************
In my TFTP root, I then extract the following RPMs into subdirectories (from Fedora 29 currently):
syslinux-tftpboot shim-x64 grub2-efi-x64 grub2-efi-x64-modules
I put each in a subdirectory and then add symlinks to make it easier to replace things with the contents of newer RPMs. I'm including a script I use that sets that up (and can be re-run at any time to download the latest RPMs - this assumes the PXE server is Fedora though, but you should be able to adapt it easily enough).
Since you already have a working BIOS PXE, I'll assume you know how to make that config file. The kernel/initrd lines take HTTP URLs just fine, so that's much faster.
Here's what the output of my grub2.pl CGI looks like (I use gfxterm so I can load a larger font that I set up locally in the TFTP root - you should be able to just skip that line):
******************** terminal_input console loadfont /12x26.pf2 insmod gfxterm set gfxmode=auto terminal_output gfxterm set timeout=-1
set timeout=-1
menuentry 'Install Fedora release 29 x86_64' { set root=(http,pxesrv.cmadams.net) linuxefi /pub/fedora/linux/releases/29/Server/x86_64/os/images/pxeboot/vmlinuz inst.root=http://pxesrv.cmadams.net/pub/fedora/linux/releases/29/Server/x86_64/os/ inst.stage2=http://pxesrv.cmadams.net/pub/fedora/linux/releases/29/Server/x86_64/os/ repo=http://pxesrv.cmadams.net/pub/fedora/linux/releases/29/Everything/x86_64/os quiet BOOTIF=$net_default_mac initrdefi /pub/fedora/linux/releases/29/Server/x86_64/os/images/pxeboot/initrd.img } ********************
Hope this helps - at least show you some ways to do things. Here's my tftpboot setup script:
******************** #!/bin/bash
# Set up a BIOS/UEFI PXE TFTP boot tree on a Fedora system set -e
# Fetch and extract the RPMs dnf download syslinux-tftpboot shim-x64 grub2-efi-x64 grub2-efi-x64-modules memtest86+ for rpm in *.rpm; do pkg=${rpm%.rpm} if [ ! -d $pkg ]; then mkdir $pkg cd $pkg rpm2cpio ../$rpm | cpio -dumi cd .. name=$(rpm -q --qf '%{name}' -p $rpm) rm -f $name ln -s $pkg $name fi rm $rpm done
# BIOS setup if [ ! -d bios ]; then mkdir bios cd bios ln -s ../syslinux-tftpboot/tftpboot/{ldlinux.c32,libutil.c32,lpxelinux.0,memdisk,menu.c32} . ln -s ../memtest86+/boot/memtest86+-* memtest-cur echo "ui menu.c32 http://pxesrv.cmadams.net/local/pxe.pl" > pxelinux.conf cd .. fi
# EFI setup if [ ! -d EFI ]; then mkdir EFI cd EFI ln -s ../grub2-efi-x64-modules/usr/lib/grub fedora cd .. ln -s shim-x64/boot/efi/EFI/fedora/shimx64.efi . ln -s grub2-efi-x64/boot/efi/EFI/fedora/grubx64.efi . echo "sournce (http,pxesrv,cmadams.net)/local/grub2.pl" > grub.cfg fi ********************
Chris Adams writes:
Once upon a time, isdtor isdtor@gmail.com said:
We have a working PXE setup. I've tried to adapt it to UEFI as per RHEL6 manual, but the client won't boot.
I have BIOS+UEFI PXE boot set up, although it took some doing. I still use SYSLINUX for BIOS, but GRUB2 plus Red Hat's secure boot shim for UEFI. I never could get SYSLINUX's UEFI support working very well or reliably, and it didn't support secure boot at all last I looked.
I also feed most of the content out via HTTP (much faster than TFTP). That includes generating menus on the fly from perl CGIs (I'm old-fashioned that way :) ).
I'm using dnsmasq as my DHCP server, so I have this in its config (where pxesrv.cmadams.net has IP 10.10.10.2):
Thanks for the detailed info. But my setup is different. ISC dhcpd is set up as per https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/htm..., and grub.
next-server points to to the tftp server but seems to be ignored. When the client boots, tcpdump reports
09:02:02.911381 IP client.cisco-ipsla > dhcp-server.tftp: 56 RRQ "linux-install/bootx64.efi" octet tsize 0 blksize 32768 09:02:02.911403 IP dhcp-server > client: ICMP dhcp-server udp port tftp unreachable, length 92
on the dhcp server and nothing on the tftp server. The dhcp server also logs that a DHCPOFFER is made and accepted by the client.
This poster had the same problem and it was not resolved. https://superuser.com/questions/1182862/uefi-mode-pxe-booting-doesnt-work
On Thu, Apr 25, 2019 at 5:25 AM isdtor isdtor@gmail.com wrote:
09:02:02.911381 IP client.cisco-ipsla > dhcp-server.tftp: 56 RRQ "linux-install/bootx64.efi" octet tsize 0 blksize 32768 09:02:02.911403 IP dhcp-server > client: ICMP dhcp-server udp port tftp unreachable, length 92
The “ICMP unreachable” should be a dead giveaway. . .
You are missing the `firewall-cmd` configuration. See this guide:
https://www.bo-yang.net/2015/08/31/centos7-install-tftp-server
Once upon a time, Steven Tardy sjt5atra@gmail.com said:
The “ICMP unreachable” should be a dead giveaway. . .
You cut out the part of the email where the OP said that the UEFI system was ignoring the next-server part of the DHCP reply and trying to TFTP to the DHCP server instead of the TFTP server. Of course that got ICMP unreachable, but it isn't a firewall problem.
To the OP: can you post a full tcpdump decode of the DHCP offer? I seem to remember having some issue with ISC DHCP in the past not getting the next server set correctly - maybe a packet will jog some memory (and I'll go try to grab the same from my dnsmasq DHCP for comparison).
On Thu, Apr 25, 2019 at 8:46 PM Chris Adams linux@cmadams.net wrote:
Once upon a time, Steven Tardy sjt5atra@gmail.com said:
The “ICMP unreachable” should be a dead giveaway. . .
You cut out the part of the email where the OP said that the UEFI system was ignoring the next-server part of the DHCP reply and trying to TFTP to the DHCP server instead of the TFTP server. Of course that got ICMP unreachable, but it isn't a firewall problem.
To the OP: can you post a full tcpdump decode of the DHCP offer? I seem to remember having some issue with ISC DHCP in the past not getting the next server set correctly - maybe a packet will jog some memory (and I'll go try to grab the same from my dnsmasq DHCP for comparison).
Just set up ISC DHCP on fresh CentOS 7 install and followed the redhat guide linked in this thread. Did what I thought was correct and duplicated the OPs problem. /me scratches head. . . off to `tcpdump -vv -nn -i ens192`. . . packets never lie. . . Vendor-Class Option 60, length 32: "PXEClient:Arch:00007:UNDI:003000"
d'oh; I did _lower case_ a string in dhcpd.conf line on accident while transcribing: match if substring (option vendor-class-identifier, 0, 9) = "pxeclient";
Changed it to follow redhat guide CAMELCase and poof, match if substring (option vendor-class-identifier, 0, 9) = "PXEClient";
PXEClient tried to reach out "next-server". 00:42:24.606544 IP 1.2.3.10.1165 > 1.2.3.2.69: 50 RRQ "pxelinux/bootx64.efi" octet tsize 0 blksize 1468
If the OP still can't get this working, packet captures really do help. (;
[root@DHCPServer ~]# cat /etc/dhcp/dhcpd.conf option arch code 93 = unsigned integer 16; #RFC 4578 subnet 1.2.3.0 netmask 255.255.255.0 { option routers 1.2.3.254; range 1.2.3.10 1.2.3.20; class "pxeclients" { match if substring (option vendor-class-identifier, 0, 9) = "PXEClient"; next-server 1.2.3.2; if option arch = 00:06 { filename = "pxelinux/bootia32.efi"; } else if option arch = 00:07 { filename = "pxelinux/bootx64.efi"; } else { filename = "pxelinux/pxelinux.0"; } } }
Just set up ISC DHCP on fresh CentOS 7 install and followed the redhat guide linked in this thread. Did what I thought was correct and duplicated the OPs problem. /me scratches head. . . off to `tcpdump -vv -nn -i ens192`. . . packets never lie. . . Vendor-Class Option 60, length 32: "PXEClient:Arch:00007:UNDI:003000"
d'oh; I did _lower case_ a string in dhcpd.conf line on accident while transcribing: match if substring (option vendor-class-identifier, 0, 9) = "pxeclient";
Ok, this is not the case here, case is correct, and tcpdump shows that the correct file is requested.
However:
11:06:51.413549 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 390) 10.1.2.2.67 > 255.255.255.255.68: [udp sum ok] BOOTP/DHCP, Reply, length 362, xid 0x4007adc6, Flags [Broadcast] (0x8000) Your-IP 10.1.2.57 Server-IP 10.1.2.1 <-- Client-Ethernet-Address 00:1b:21:d8:69:1c file "linux-install/bootx64.efi" Vendor-rfc1048 Extensions Magic Cookie 0x63825363 DHCP-Message Option 53, length 1: ACK Server-ID Option 54, length 4: 10.1.2.2 Lease-Time Option 51, length 4: 43200 Subnet-Mask Option 1, length 4: 255.255.255.0 Default-Gateway Option 3, length 4: 10.1.2.250 Domain-Name-Server Option 6, length 8: 10.1.2.2 Hostname Option 12, length 5: "client" Domain-Name Option 15, length 20: "foo.bar.com" NTP Option 42, length 8: 10.1.2.2 RN Option 58, length 4: 21600 RB Option 59, length 4: 37800 TFTP Option 66, length 11: "10.1.2.1" <-- END Option 255, length 0 11:06:51.413552 IP (tos 0x0, ttl 64, id 64565, offset 0, flags [none], proto UDP (17), length 84) 10.1.2.57.1943 > 10.1.2.2.69: [udp sum ok] 56 RRQ "linux-install/bootx64.efi" octet tsize 0 blksize 32768 <--
Where: dhcp server = 10.1.2.2, tftp server 10.1.2.1, client 10.1.2.57. DHCP answer is all good and the client still requests the wrong server
I will check if there are any NIC fw updates. If not, the workaround is to move the tftp server to the dhcp server.
And I thought computers were supposed to do my bidding and not the other way round ...
Once upon a time, isdtor isdtor@gmail.com said:
11:06:51.413549 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 390) 10.1.2.2.67 > 255.255.255.255.68: [udp sum ok] BOOTP/DHCP, Reply, length 362, xid 0x4007adc6, Flags [Broadcast] (0x8000) Your-IP 10.1.2.57 Server-IP 10.1.2.1 <-- Client-Ethernet-Address 00:1b:21:d8:69:1c file "linux-install/bootx64.efi" Vendor-rfc1048 Extensions Magic Cookie 0x63825363 DHCP-Message Option 53, length 1: ACK Server-ID Option 54, length 4: 10.1.2.2 Lease-Time Option 51, length 4: 43200 Subnet-Mask Option 1, length 4: 255.255.255.0 Default-Gateway Option 3, length 4: 10.1.2.250 Domain-Name-Server Option 6, length 8: 10.1.2.2 Hostname Option 12, length 5: "client" Domain-Name Option 15, length 20: "foo.bar.com" NTP Option 42, length 8: 10.1.2.2 RN Option 58, length 4: 21600 RB Option 59, length 4: 37800 TFTP Option 66, length 11: "10.1.2.1" <-- END Option 255, length 0
I do see a couple of differences - main one is that my boot file is in option 67, not the BOOTP "file" field. Also, my option 66 is a hostname, not an IP. I don't know how you tell ISC DHCP to use option 67 instead of the file field, but maybe that could trigger different client behavior?
More odd is that dnsmasq is adding a null terminator to both options 66 and 67. My UEFI PXE clients seem to accept it just fine though.
Chris Adams writes:
Once upon a time, isdtor isdtor@gmail.com said:
11:06:51.413549 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 390) 10.1.2.2.67 > 255.255.255.255.68: [udp sum ok] BOOTP/DHCP, Reply, length 362, xid 0x4007adc6, Flags [Broadcast] (0x8000) Your-IP 10.1.2.57 Server-IP 10.1.2.1 <-- Client-Ethernet-Address 00:1b:21:d8:69:1c file "linux-install/bootx64.efi" Vendor-rfc1048 Extensions Magic Cookie 0x63825363 DHCP-Message Option 53, length 1: ACK Server-ID Option 54, length 4: 10.1.2.2 Lease-Time Option 51, length 4: 43200 Subnet-Mask Option 1, length 4: 255.255.255.0 Default-Gateway Option 3, length 4: 10.1.2.250 Domain-Name-Server Option 6, length 8: 10.1.2.2 Hostname Option 12, length 5: "client" Domain-Name Option 15, length 20: "foo.bar.com" NTP Option 42, length 8: 10.1.2.2 RN Option 58, length 4: 21600 RB Option 59, length 4: 37800 TFTP Option 66, length 11: "10.1.2.1" <-- END Option 255, length 0
I do see a couple of differences - main one is that my boot file is in option 67, not the BOOTP "file" field. Also, my option 66 is a hostname, not an IP. I don't know how you tell ISC DHCP to use option 67 instead of the file field, but maybe that could trigger different client behavior?
More odd is that dnsmasq is adding a null terminator to both options 66 and 67. My UEFI PXE clients seem to accept it just fine though.
Yes, it looks like I'm out of luck and need to find a newer machine to test this with. Moving the tftp server works to an extent - server boots right into a grub prompt.
I went over the linux-poweredge archives and found this:
https://lists.us.dell.com/pipermail/linux-poweredge/2015-May/049810.html
In particular: | 2. Don't waste time w/ a R*10, the UEFI PXE boot code is buggy! It misinterprets the NBP filename (DHCP option 67). That's an old Intel bug; they fixed it years ago in their BIOS PXE implementation. I'm guessing it was resurrected in their UEFI PXE implementation. | | Most all NIC vendors (Intel, Broadcom, etc) use the Intel reference implementation for PXE.
Once upon a time, isdtor isdtor@gmail.com said:
Yes, it looks like I'm out of luck and need to find a newer machine to test this with. Moving the tftp server works to an extent - server boots right into a grub prompt.
Weird. I have a couple of Intel-based systems that do UEFI PXE boot okay, but they're much newer than the 11th gen PowerEdge (I have some of those still in service but they're not running UEFI mode). Also, they're client-type systems, not servers (a Thinkpad notebook and an Intel NUC), so possibly different firmware base.
I vaguely remember in the distant past having some kind of problem like this, and I think on those systems, I used a boot floppy (yes, that long ago!) with gPXE on it.
Chris Adams writes:
Once upon a time, isdtor isdtor@gmail.com said:
Yes, it looks like I'm out of luck and need to find a newer machine to test this with. Moving the tftp server works to an extent - server boots right into a grub prompt.
Weird. I have a couple of Intel-based systems that do UEFI PXE boot okay, but they're much newer than the 11th gen PowerEdge (I have some of those still in service but they're not running UEFI mode). Also, they're client-type systems, not servers (a Thinkpad notebook and an Intel NUC), so possibly different firmware base.
I vaguely remember in the distant past having some kind of problem like this, and I think on those systems, I used a boot floppy (yes, that long ago!) with gPXE on it.
Not too happy to revive this old thread ...
Has anyone managed to UEFI/PXE boot CentOS6? I managed to find newer hw and have this working perfectly with CentOS7. With 6, however, I again cannot boot beyond a grub prompt.
CentOS7 bootx64.efi does not work for CentOS6 as it complains "kernel too old". Booting with CentOS6 /boot/efi/EFI/redhat/grub.efi, and the older style pxe config file as per https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/htm... results in a grub (legacy) prompt on the target machine.
On 10/21/19 6:38 AM, isdtor wrote:
Booting with CentOS6 /boot/efi/EFI/redhat/grub.efi, and the older style pxe config file as per ... results in a grub (legacy) prompt on the target machine.
Have you tried using the file from the installation tree?
http://centos.s.uw.edu/centos/6/os/x86_64/EFI/BOOT/
or with files extracted from one of the ISOs?
Gordon Messmer writes:
On 10/21/19 6:38 AM, isdtor wrote:
Booting with CentOS6 /boot/efi/EFI/redhat/grub.efi, and the older style pxe config file as per ... results in a grub (legacy) prompt on the target machine.
Have you tried using the file from the installation tree?
http://centos.s.uw.edu/centos/6/os/x86_64/EFI/BOOT/
or with files extracted from one of the ISOs?
They are all identical to the installed one (distributed with grub).
isdtor writes:
Gordon Messmer writes:
On 10/21/19 6:38 AM, isdtor wrote:
Booting with CentOS6 /boot/efi/EFI/redhat/grub.efi, and the older style pxe config file as per ... results in a grub (legacy) prompt on the target machine.
Have you tried using the file from the installation tree?
http://centos.s.uw.edu/centos/6/os/x86_64/EFI/BOOT/
or with files extracted from one of the ISOs?
They are all identical to the installed one (distributed with grub).
I'm tantalizingly close. CentOS6 UEFI PXE is working, but the machine won't boot.
It is best to use the CentOS7 BOOTX64.EFI loader. The "kernel too old" message is due to the fact that the PXE config file uses linuxefi and initrdefi directives. They need to be changed to linux/initrd, respectively. linux16/initrd16 may also work. The arguments to those directives need to adhere to CentOS6 legacy grub syntax (ks instead of inst.ks etc.).
This will cause error messages on the client that both linux and initrd are unknown commands. The fix is to add the linux module to the PXE config file,
insmod linux
The client now issues different error messages that flash past pretty quickly. tcpdump to the rescue, it becomes clear that it is seeking to download the grub linux module through tftp, so I added the complete contents of the CentOS7 grub2-efi-x64-modules rpm under /tftpboot/EFI/centos/x86_64-efi.
These changes will lead to a successful PXE boot. If a kickstart installation is used, one also needs to add a /boot/efi partition. Looking at a CentOS7 machine, type vfat and size 200MB should be fine; reqpart is not support under CentOS6. Last, and this untested as of yet, it looks like the bootloader location needs to be partition because the resulting disk is in GPT format.
Cracked it.
Failure to boot was the result of including tboot in the install package set, for some reason or other. It's not needed and removing it does the trick.
https://access.redhat.com/articles/2217041
And I thought I was going spare when it worked with one install package set and not another ...
When using a kickstart file, the bootloader, clearpart and zerombr directives also need attention. Used without the correct options they will install an MBR which is not exactly desired.
-
Chris Adams -
Gordon Messmer -
isdtor -
Steven Tardy