kickstart: dracut-initqueue fails due to unresolvable hostname even though network config looks perfectly ok
Hi,
kickstarting fails due to problems with host resolution, even though the network seems to be properly configured through DHCP. eno1 and eno2 are both attached to the network, but only eno1 gets an IP via DHCP. Still `curl` cannot resolve the mirror host and the kickstart host during dracut-initqueue:
rdsosreport.txt ------------------------------------------------ [...] [ 14.780428] localhost kernel: IPv6: ADDRCONF(NETDEV_UP): eno1: link is not ready [ 19.977052] localhost kernel: tg3 0000:0b:00.0 eno1: Link is up at 1000 Mbps, full duplex [ 19.977118] localhost kernel: tg3 0000:0b:00.0 eno1: Flow control is off for TX and off for RX [ 19.978880] localhost kernel: tg3 0000:0b:00.0 eno1: EEE is disabled [ 19.980693] localhost kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eno1: link becomes ready [ 19.829468] localhost dracut-initqueue[992]: dhcp: PREINIT eno1 up [ 19.853734] localhost dhclient[1393]: DHCPDISCOVER on eno1 to 255.255.255.255 port 67 interval 6 (xid=0x4df19201) [ 26.030151] localhost dhclient[1393]: DHCPDISCOVER on eno1 to 255.255.255.255 port 67 interval 13 (xid=0x4df19201) [ 26.033472] localhost dhclient[1393]: DHCPREQUEST on eno1 to 255.255.255.255 port 67 (xid=0x4df19201) [ 26.033668] localhost dhclient[1393]: DHCPOFFER from 10.128.196.98 [ 26.038851] localhost dhclient[1393]: DHCPACK from 10.128.196.98 (xid=0x4df19201) [ 26.067534] localhost dracut-initqueue[992]: dhcp: BOND setting eno1 [ 28.082735] localhost dhclient[1393]: bound to 10.128.196.20 -- renewal in 21301 seconds. [ 28.456131] localhost kernel: tg3 0000:0b:00.1: irq 153 for MSI/MSI-X [ 28.456149] localhost kernel: tg3 0000:0b:00.1: irq 154 for MSI/MSI-X [ 28.456165] localhost kernel: tg3 0000:0b:00.1: irq 155 for MSI/MSI-X [ 28.456180] localhost kernel: tg3 0000:0b:00.1: irq 156 for MSI/MSI-X [ 28.456196] localhost kernel: tg3 0000:0b:00.1: irq 157 for MSI/MSI-X [ 28.570450] localhost kernel: IPv6: ADDRCONF(NETDEV_UP): eno2: link is not ready [ 34.024621] localhost kernel: tg3 0000:0b:00.1 eno2: Link is up at 1000 Mbps, full duplex [ 34.026347] localhost kernel: tg3 0000:0b:00.1 eno2: Flow control is off for TX and off for RX [ 34.028069] localhost kernel: tg3 0000:0b:00.1 eno2: EEE is disabled [ 34.029776] localhost kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eno2: link becomes ready [ 33.803606] localhost dracut-initqueue[992]: dhcp: PREINIT eno2 up [ 33.827664] localhost dhclient[1570]: DHCPDISCOVER on eno2 to 255.255.255.255 port 67 interval 7 (xid=0x1e8bdc4b) [ 41.000199] localhost dhclient[1570]: DHCPDISCOVER on eno2 to 255.255.255.255 port 67 interval 15 (xid=0x1e8bdc4b) [ 55.588353] localhost dhclient[1570]: DHCPDISCOVER on eno2 to 255.255.255.255 port 67 interval 18 (xid=0x1e8bdc4b) [ 74.172423] localhost dhclient[1570]: DHCPDISCOVER on eno2 to 255.255.255.255 port 67 interval 12 (xid=0x1e8bdc4b) [ 86.446514] localhost dhclient[1570]: DHCPDISCOVER on eno2 to 255.255.255.255 port 67 interval 9 (xid=0x1e8bdc4b) [ 95.253443] localhost dhclient[1570]: No DHCPOFFERS received. [ 95.253648] localhost dhclient[1570]: No working leases in persistent database - sleeping. [ 95.282175] localhost dracut-initqueue[992]: dhcp: FAIL [ 95.353255] localhost dracut-initqueue[992]: RTNETLINK answers: File exists [ 102.502688] localhost dracut-initqueue[992]: Warning: can't find installer mainimage path in .treeinfo [ 102.517568] localhost dracut-initqueue[992]: % Total % Received % Xferd Average Speed Time Time Time Current [ 102.525942] localhost dracut-initqueue[992]: Dload Upload Total Spent Left Speed [ 102.534277] localhost dracut-initqueue[992]: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Warning: Transient problem: timeout Will retry in 1 seconds. 3 retries left. [ 103.527190] localhost dracut-initqueue[992]: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Warning: Transient problem: timeout Will retry in 2 seconds. 2 retries left. [ 105.533677] localhost dracut-initqueue[992]: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Warning: Transient problem: timeout Will retry in 4 seconds. 1 retries left. [ 109.542329] localhost dracut-initqueue[992]: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (6) Could not resolve host: our.centos.mirror; Unknown error [ 109.551613] localhost dracut-initqueue[992]: Warning: Downloading 'http://our.centos.mirror/7.2.1511/os/x86_64/LiveOS/squashfs.img' failed! [ 109.615143] localhost dracut-initqueue[992]: % Total % Received % Xferd Average Speed Time Time Time Current [ 109.624698] localhost dracut-initqueue[992]: Dload Upload Total Spent Left Speed [ 109.625018] localhost dracut-initqueue[992]: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Warning: Transient problem: timeout Will retry in 1 seconds. 3 retries left. [ 110.624800] localhost dracut-initqueue[992]: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Warning: Transient problem: timeout Will retry in 2 seconds. 2 retries left. [ 112.631351] localhost dracut-initqueue[992]: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Warning: Transient problem: timeout Will retry in 4 seconds. 1 retries left. [ 116.639935] localhost dracut-initqueue[992]: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (6) Could not resolve host: our.kickstart.server; Unknown error [ 116.650727] localhost dracut-initqueue[992]: Warning: failed to fetch kickstart from http://our.kickstart.server/ks.cgi [ 242.126427] our.client dracut-initqueue[992]: Warning: dracut-initqueue timeout - starting timeout scripts [... 120 times this warning ...] [ 304.648460] our.client systemd[1]: Received SIGRTMIN+20 from PID 991 (plymouthd). [ 304.650862] our.client dracut-initqueue[992]: Warning: /dev/root does not exist [ 304.659840] our.client systemd[1]: Starting Dracut Emergency Shell... [ 304.684459] our.client systemd[1]: Received SIGRTMIN+21 from PID 991 (plymouthd). [... exiting to emergency shell ...]
dropped to the emergency shell, the network config looks absolutely ok:
$ ip route default via 10.128.196.97 dev eno1 10.128.196.0/24 dev eno1 proto kernel scope link src 10.128.196.20 169.254.95.0/24 dev enp0s20u1u5 proto kernel scope link src 169.254.95.120 $
`ip a` shows the correct IP address for eno1 and /etc/resolv.conf has all required nameservers. From the emergency shell both hostnames (our.centos.mirror and our.kickstart.server) are perfectly resolvable and accessible.
The used PXE config is:
LABEL CentOS-7 kernel centos-7.2/vmlinuz append initrd=centos-7.2/initrd.img ip=dhcp inst.repo=http://our.centos.mirror/7.2.1511/os/x86_64 inst.ks.sendmac inst.ks=http://our.kickstart.server/ks.cgi
Any idea, why curl cannot resolve the two hostnames or how I can debug this further?
Cheers frank
This problem still bites us. I've tried to play around with DHCP settings (rd.net.timeout.dhcp, rd.net.dhcp.retry) to no avail.
I'm happy about /any/ hint.
Cheers frank
On 04/24/2017 11:57 PM, Frank Thommen wrote:
Hi,
kickstarting fails due to problems with host resolution, even though the network seems to be properly configured through DHCP. eno1 and eno2 are both attached to the network, but only eno1 gets an IP via DHCP. Still `curl` cannot resolve the mirror host and the kickstart host during dracut-initqueue:
rdsosreport.txt
[...] [ 14.780428] localhost kernel: IPv6: ADDRCONF(NETDEV_UP): eno1: link is not ready [ 19.977052] localhost kernel: tg3 0000:0b:00.0 eno1: Link is up at 1000 Mbps, full duplex [ 19.977118] localhost kernel: tg3 0000:0b:00.0 eno1: Flow control is off for TX and off for RX [ 19.978880] localhost kernel: tg3 0000:0b:00.0 eno1: EEE is disabled [ 19.980693] localhost kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eno1: link becomes ready [ 19.829468] localhost dracut-initqueue[992]: dhcp: PREINIT eno1 up [ 19.853734] localhost dhclient[1393]: DHCPDISCOVER on eno1 to 255.255.255.255 port 67 interval 6 (xid=0x4df19201) [ 26.030151] localhost dhclient[1393]: DHCPDISCOVER on eno1 to 255.255.255.255 port 67 interval 13 (xid=0x4df19201) [ 26.033472] localhost dhclient[1393]: DHCPREQUEST on eno1 to 255.255.255.255 port 67 (xid=0x4df19201) [ 26.033668] localhost dhclient[1393]: DHCPOFFER from 10.128.196.98 [ 26.038851] localhost dhclient[1393]: DHCPACK from 10.128.196.98 (xid=0x4df19201) [ 26.067534] localhost dracut-initqueue[992]: dhcp: BOND setting eno1 [ 28.082735] localhost dhclient[1393]: bound to 10.128.196.20 -- renewal in 21301 seconds. [ 28.456131] localhost kernel: tg3 0000:0b:00.1: irq 153 for MSI/MSI-X [ 28.456149] localhost kernel: tg3 0000:0b:00.1: irq 154 for MSI/MSI-X [ 28.456165] localhost kernel: tg3 0000:0b:00.1: irq 155 for MSI/MSI-X [ 28.456180] localhost kernel: tg3 0000:0b:00.1: irq 156 for MSI/MSI-X [ 28.456196] localhost kernel: tg3 0000:0b:00.1: irq 157 for MSI/MSI-X [ 28.570450] localhost kernel: IPv6: ADDRCONF(NETDEV_UP): eno2: link is not ready [ 34.024621] localhost kernel: tg3 0000:0b:00.1 eno2: Link is up at 1000 Mbps, full duplex [ 34.026347] localhost kernel: tg3 0000:0b:00.1 eno2: Flow control is off for TX and off for RX [ 34.028069] localhost kernel: tg3 0000:0b:00.1 eno2: EEE is disabled [ 34.029776] localhost kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eno2: link becomes ready [ 33.803606] localhost dracut-initqueue[992]: dhcp: PREINIT eno2 up [ 33.827664] localhost dhclient[1570]: DHCPDISCOVER on eno2 to 255.255.255.255 port 67 interval 7 (xid=0x1e8bdc4b) [ 41.000199] localhost dhclient[1570]: DHCPDISCOVER on eno2 to 255.255.255.255 port 67 interval 15 (xid=0x1e8bdc4b) [ 55.588353] localhost dhclient[1570]: DHCPDISCOVER on eno2 to 255.255.255.255 port 67 interval 18 (xid=0x1e8bdc4b) [ 74.172423] localhost dhclient[1570]: DHCPDISCOVER on eno2 to 255.255.255.255 port 67 interval 12 (xid=0x1e8bdc4b) [ 86.446514] localhost dhclient[1570]: DHCPDISCOVER on eno2 to 255.255.255.255 port 67 interval 9 (xid=0x1e8bdc4b) [ 95.253443] localhost dhclient[1570]: No DHCPOFFERS received. [ 95.253648] localhost dhclient[1570]: No working leases in persistent database - sleeping. [ 95.282175] localhost dracut-initqueue[992]: dhcp: FAIL [ 95.353255] localhost dracut-initqueue[992]: RTNETLINK answers: File exists [ 102.502688] localhost dracut-initqueue[992]: Warning: can't find installer mainimage path in .treeinfo [ 102.517568] localhost dracut-initqueue[992]: % Total % Received % Xferd Average Speed Time Time Time Current [ 102.525942] localhost dracut-initqueue[992]: Dload Upload Total Spent Left Speed [ 102.534277] localhost dracut-initqueue[992]: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Warning: Transient problem: timeout Will retry in 1 seconds. 3 retries left. [ 103.527190] localhost dracut-initqueue[992]: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Warning: Transient problem: timeout Will retry in 2 seconds. 2 retries left. [ 105.533677] localhost dracut-initqueue[992]: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Warning: Transient problem: timeout Will retry in 4 seconds. 1 retries left. [ 109.542329] localhost dracut-initqueue[992]: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (6) Could not resolve host: our.centos.mirror; Unknown error [ 109.551613] localhost dracut-initqueue[992]: Warning: Downloading 'http://our.centos.mirror/7.2.1511/os/x86_64/LiveOS/squashfs.img' failed! [ 109.615143] localhost dracut-initqueue[992]: % Total % Received % Xferd Average Speed Time Time Time Current [ 109.624698] localhost dracut-initqueue[992]: Dload Upload Total Spent Left Speed [ 109.625018] localhost dracut-initqueue[992]: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Warning: Transient problem: timeout Will retry in 1 seconds. 3 retries left. [ 110.624800] localhost dracut-initqueue[992]: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Warning: Transient problem: timeout Will retry in 2 seconds. 2 retries left. [ 112.631351] localhost dracut-initqueue[992]: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Warning: Transient problem: timeout Will retry in 4 seconds. 1 retries left. [ 116.639935] localhost dracut-initqueue[992]: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (6) Could not resolve host: our.kickstart.server; Unknown error [ 116.650727] localhost dracut-initqueue[992]: Warning: failed to fetch kickstart from http://our.kickstart.server/ks.cgi [ 242.126427] our.client dracut-initqueue[992]: Warning: dracut-initqueue timeout - starting timeout scripts [... 120 times this warning ...] [ 304.648460] our.client systemd[1]: Received SIGRTMIN+20 from PID 991 (plymouthd). [ 304.650862] our.client dracut-initqueue[992]: Warning: /dev/root does not exist [ 304.659840] our.client systemd[1]: Starting Dracut Emergency Shell... [ 304.684459] our.client systemd[1]: Received SIGRTMIN+21 from PID 991 (plymouthd). [... exiting to emergency shell ...]
dropped to the emergency shell, the network config looks absolutely ok:
$ ip route default via 10.128.196.97 dev eno1 10.128.196.0/24 dev eno1 proto kernel scope link src 10.128.196.20 169.254.95.0/24 dev enp0s20u1u5 proto kernel scope link src 169.254.95.120 $
`ip a` shows the correct IP address for eno1 and /etc/resolv.conf has all required nameservers. From the emergency shell both hostnames (our.centos.mirror and our.kickstart.server) are perfectly resolvable and accessible.
The used PXE config is:
LABEL CentOS-7 kernel centos-7.2/vmlinuz append initrd=centos-7.2/initrd.img ip=dhcp inst.repo=http://our.centos.mirror/7.2.1511/os/x86_64 inst.ks.sendmac inst.ks=http://our.kickstart.server/ks.cgi
Any idea, why curl cannot resolve the two hostnames or how I can debug this further?
Cheers frank
CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Hi,
On Mon, May 15, 2017 at 03:04:03PM +0200, Frank Thommen wrote:
This problem still bites us. I've tried to play around with DHCP settings (rd.net.timeout.dhcp, rd.net.dhcp.retry) to no avail.
I'm happy about /any/ hint.
1) ip route seems ok, but what does your /etc/resolv.conf looks like?
2) could you try with 7.3.1611, 7.2.1511 is no longer supported.
Tru
0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (6) Could not resolve host: our.centos.mirror; Unknown error
curl: (6) Could not resolve host: our.centos.mirror; Unknown error
Hi
On 15/05/17 19:30, Tru Huynh wrote:
Hi,
On Mon, May 15, 2017 at 03:04:03PM +0200, Frank Thommen wrote:
This problem still bites us. I've tried to play around with DHCP settings (rd.net.timeout.dhcp, rd.net.dhcp.retry) to no avail.
I'm happy about /any/ hint.
- ip route seems ok, but what does your /etc/resolv.conf looks like?
/etc/resolv.conf is 100% ok once I am in the dracut emergency shell and name resolution works fine. However I don't know how it looks at the time of the error message - that is during dracut-initqueue - and I have no idea how I could check that.
- could you try with 7.3.1611, 7.2.1511 is no longer supported.
Unfortunateyl not. Due to special hardware (Infiniband, FPGA) and special lustre drivers we are currently bound to this specific release.
frank
On Mon, 15 May 2017 at 20:05 zulu, Frank Thommen wrote:
lustre driver
https://downloads.hpdd.intel.com/public/lustre/latest-feature-release/el7.3....
On 05/16/2017 12:25 AM, Darr247 wrote:
On Mon, 15 May 2017 at 20:05 zulu, Frank Thommen wrote:
lustre driver
https://downloads.hpdd.intel.com/public/lustre/latest-feature-release/el7.3....
We have a special appliance attached and get the lustre driver from the appliance vendor to ensure maximum compatibility and maintenance. So https://downloads.hpdd.intel.com/public/lustre/latest-feature-release/el7.3.... unfortunately doesn't apply to us.
An then there are still FPGA and Infiniband where the vendors of our hardware currently don't support CentOS 7.3 (yet).
frank
-
Darr247 -
Frank Thommen -
Tru Huynh