I have a Dell server that has two built-in ethernet devices. When I kickstart the machine, they are correctly identified as eth0 and eth1 (correctly meaning they correspond to the physical device ports 1 and 2). I need a third one and want that to come up as eth2. After adding the hardware, kickstart now fails because for some reason it goes through a rename process where it makes the newly added card eth1 (or eth0, I forgot). Is there a way to stop this rename process so kickstart correctly uses the physical hardware the way they are, meaning physical port 1 = eth0, port 2 = eth1, and the additional ethernet card then becomes eth2?
Should I be using the device's MAC address when I set the 'network' option in the kickstart file? So instead of 'network --device=eth0' I make it 'network -device=aa;bb:cc:dd:eee:ff' ?
Correction to my own post: I know it's not kickstart that's doing the renaming, it's the kernel that's booting up the system. On Feb 23, 2015 4:34 PM, "Ashley M. Kirchner" ashley@pcraft.com wrote:
I have a Dell server that has two built-in ethernet devices. When I kickstart the machine, they are correctly identified as eth0 and eth1 (correctly meaning they correspond to the physical device ports 1 and 2). I need a third one and want that to come up as eth2. After adding the hardware, kickstart now fails because for some reason it goes through a rename process where it makes the newly added card eth1 (or eth0, I forgot). Is there a way to stop this rename process so kickstart correctly uses the physical hardware the way they are, meaning physical port 1 = eth0, port 2 = eth1, and the additional ethernet card then becomes eth2?
Should I be using the device's MAC address when I set the 'network' option in the kickstart file? So instead of 'network --device=eth0' I make it 'network -device=aa;bb:cc:dd:eee:ff' ?
On 02/23/2015 05:34 PM, Ashley M. Kirchner wrote:
I have a Dell server that has two built-in ethernet devices. When I kickstart the machine, they are correctly identified as eth0 and eth1 (correctly meaning they correspond to the physical device ports 1 and 2). I need a third one and want that to come up as eth2. After adding the hardware, kickstart now fails because for some reason it goes through a rename process where it makes the newly added card eth1 (or eth0, I forgot). Is there a way to stop this rename process so kickstart correctly uses the physical hardware the way they are, meaning physical port 1 = eth0, port 2 = eth1, and the additional ethernet card then becomes eth2?
What version of CentOS are you trying to install? I would expect that a recent version would use the biosdevname interface naming scheme on a Dell server.
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/htm...
6.6-x86_64 ...
Keep in mind this is happening during the kickstart process. What I'm seeing is this:
When the system first comes on, PXE kicks in and port 2 (or eth1) gets an IP from the network's dhcp and is configured properly. PXE grabs the necessary files to start installation. However, as soon as it the install image is uncompressed it does a rename on the devices and what should be eth3 is now configured as either eth0 or eth1 (I forgot which one) and what *should* be eth1 is effectively no longer configured. At that point kickstart stops because it can no longer fetch the kickstart file. I have to physically remove the additional ethernet card for it to work properly.
On Mon, Feb 23, 2015 at 5:48 PM, Ian Pilcher arequipeno@gmail.com wrote:
On 02/23/2015 05:34 PM, Ashley M. Kirchner wrote:
I have a Dell server that has two built-in ethernet devices. When I kickstart the machine, they are correctly identified as eth0 and eth1 (correctly meaning they correspond to the physical device ports 1 and 2). I need a third one and want that to come up as eth2. After adding the hardware, kickstart now fails because for some reason it goes through a rename process where it makes the newly added card eth1 (or eth0, I forgot). Is there a way to stop this rename process so kickstart correctly uses the physical hardware the way they are, meaning physical port 1 = eth0, port 2 = eth1, and the additional ethernet card then becomes eth2?
What version of CentOS are you trying to install? I would expect that a recent version would use the biosdevname interface naming scheme on a Dell server.
https://access.redhat.com/documentation/en-US/Red_Hat_ Enterprise_Linux/7/html/Networking_Guide/sec-Consistent_Network_Device_ Naming_Using_biosdevname.html
--
Ian Pilcher arequipeno@gmail.com
-------- "I grew up before Mark Zuckerberg invented friendship" --------
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On Feb 23, 2015, at 6:34 PM, Ashley M. Kirchner ashley@pcraft.com wrote:
I have a Dell server that has two built-in ethernet devices. When I kickstart the machine, they are correctly identified as eth0 and eth1 (correctly meaning they correspond to the physical device ports 1 and 2). I need a third one and want that to come up as eth2. After adding the hardware, kickstart now fails because for some reason it goes through a rename process where it makes the newly added card eth1 (or eth0, I forgot). Is there a way to stop this rename process so kickstart correctly uses the physical hardware the way they are, meaning physical port 1 = eth0, port 2 = eth1, and the additional ethernet card then becomes eth2?
Should I be using the device's MAC address when I set the 'network' option in the kickstart file? So instead of 'network --device=eth0' I make it 'network -device=aa;bb:cc:dd:eee:ff' ?
kickstart has an option: ksdevice=bootif
I think that'll let you accomplish what you are trying.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 23/02/15 08:16 PM, Steven Tardy wrote:
On Feb 23, 2015, at 6:34 PM, Ashley M. Kirchner ashley@pcraft.com wrote:
I have a Dell server that has two built-in ethernet devices. When I kickstart the machine, they are correctly identified as eth0 and eth1 (correctly meaning they correspond to the physical device ports 1 and 2). I need a third one and want that to come up as eth2. After adding the hardware, kickstart now fails because for some reason it goes through a rename process where it makes the newly added card eth1 (or eth0, I forgot). Is there a way to stop this rename process so kickstart correctly uses the physical hardware the way they are, meaning physical port 1 = eth0, port 2 = eth1, and the additional ethernet card then becomes eth2?
Should I be using the device's MAC address when I set the 'network' option in the kickstart file? So instead of 'network --device=eth0' I make it 'network -device=aa;bb:cc:dd:eee:ff' ?
kickstart has an option: ksdevice=bootif
I think that'll let you accomplish what you are trying.
Totally unrelated, but this is the reason I love discussions like this getting into the archives. I had no idea this option existed and it just solved an annoying problems I've been trying to think how to solve for ages!
In PXE's 'default';
LABEL new-node1 MENU LABEL ^1) New Node 1 - RHEL 6 KERNEL boot/rhel6/x86_64/vmlinuz IPAPPEND 2 APPEND initrd=boot/rhel6/x86_64/initrd.img ks=http://10.20.4.1/rhel6/x86_64/ks/pxe-ccrs-node2.ks ksdevice=bootif
Then in kickstart;
network --bootproto dhcp --onboot yes --hostname node1.example.com
(not the lack of --device)
With this, my nodes with 6 NICs reliably boot without asking the user to choose the NIC by MAC they want to install from.
Thanks!!
- -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education?
Ok, so some of this now works, but I'm still having problems. With the bootif option, the system now correctly configures and uses the same interface to get its kickstart file. However, when the system is done and boots up, the interfaces are still messed up. So this is what I have in the kickstart file:
# On-Board Port 1 with public IP configuration network --noipv6 --onboot yes --bootproto static --device eth0 --ip x.x.x.x --netmask y.y.y.y --gateway z.z.z.z # On-Board Port 2 on private subnet A network --noipv6 --onboot yes --bootproto dhcp --hostname portico # Ethernet Card on private subnet B network --noipv6 --onboot yes --bootproto dhcp --device eth2
In the PXE config file I have:
IPAPPEND 2 APPEND ks=http://192.168.x.x/ks/portico.ks initrd=centos/x86_64/initrd.img ramdisk_size=100000 ksdevice=bootif
When the system comes up and PXE kicks in, it will get an IP on its second port (eth1) to the 192.168.x.x subnet where the kickstart and install files are. When the system is completely done and boots back up, it should have: port 1 (eth0) with the static public IP assigned from the ks file port 2 (eth1) with the same DHCP assigned ip that the PXE originally received And eth2, which is an additional ethernet card will get another IP from a different dhcp server on a separate subnet.
What's happening now is, PXE gets an IP, the system will successfully get the kickstart file and go through the full setup process, however when it reboots, this is what I end up with as far as the ethernet ports:
The additional ethernet card is configured as eth0, with the public IP that the kickstart file has in it port 1 (now eth1 instead of eth0) is configured with a 10.1.10.10 IP ... Whiskey Tango Foxtrot?? port 2 (now eth2 instead of eth1) is configured with the dhcp IP that it was given during the setup.
This results in nothing working as the ports are wired into specific routers and if the boot process renames/reshuffles them, I'm left with a machine I can not get on to and that doesn't work on the network.
As soon as I *remove* the additional ethernet card, the system will boot up with the ports configured correctly (port 1 = eth0, port 2 = eth1). So why is it that as soon as there is an additional one, all things go to hell? Why must the boot process shuffle them? More importantly, how do I prevent this so that the system comes up properly after a kickstart install?
On Tue, Feb 24, 2015 at 6:47 PM, Digimer lists@alteeve.ca wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 23/02/15 08:16 PM, Steven Tardy wrote:
On Feb 23, 2015, at 6:34 PM, Ashley M. Kirchner ashley@pcraft.com wrote:
I have a Dell server that has two built-in ethernet devices. When I kickstart the machine, they are correctly identified as eth0 and eth1 (correctly meaning they correspond to the physical device ports 1 and 2). I need a third one and want that to come up as eth2. After adding the hardware, kickstart now fails because for some reason it goes through a rename process where it makes the newly added card eth1 (or eth0, I forgot). Is there a way to stop this rename process so kickstart correctly uses the physical hardware the way they are, meaning physical port 1 = eth0, port 2 = eth1, and the additional ethernet card then becomes eth2?
Should I be using the device's MAC address when I set the 'network' option in the kickstart file? So instead of 'network --device=eth0' I make it 'network -device=aa;bb:cc:dd:eee:ff' ?
kickstart has an option: ksdevice=bootif
I think that'll let you accomplish what you are trying.
Totally unrelated, but this is the reason I love discussions like this getting into the archives. I had no idea this option existed and it just solved an annoying problems I've been trying to think how to solve for ages!
In PXE's 'default';
LABEL new-node1 MENU LABEL ^1) New Node 1 - RHEL 6 KERNEL boot/rhel6/x86_64/vmlinuz IPAPPEND 2 APPEND initrd=boot/rhel6/x86_64/initrd.img ks=http://10.20.4.1/rhel6/x86_64/ks/pxe-ccrs-node2.ks ksdevice=bootif
Then in kickstart;
network --bootproto dhcp --onboot yes --hostname node1.example.com
(not the lack of --device)
With this, my nodes with 6 NICs reliably boot without asking the user to choose the NIC by MAC they want to install from.
Thanks!!
Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? -----BEGIN PGP SIGNATURE----- Version: GnuPG v1
iQIcBAEBAgAGBQJU7SnDAAoJECChztQA3mh0+dEQAMWM705Tc9fWr/ODiLDQNQHk 5todiurUcM72zPn3NCwiLTb/ZEXbnkL74Zy7qQPf8zzFryLIuldGMDIVIgVp5k3m LnkU9dW0zguXnCfde3gXJs8taYSAYA/ciwO9mE+M3V4+VU6TvzjPkVxKGkhTxjTL 5/DBz1N9V6IChRLbjcQbkHJD5gAPY0cloOoP6f0FC/k+Ojeo7oUibYQjVB8nDkwa cfxxJ2yYIjOkTBm7vQuLnHf64jR8siqN9Zw5gZuuTBfbK2gIuMw99Fg7/QAEe85h uQttjHloI1SfhYN4D5AuQzeXFXTUM3IIkRr4KzGCmKezGi4s+wDrhm4goNmsOuiH ruf80gDjW+PZADx2Q4GHPpCRe2sCyLXDFPdUrvooCLrInXFK1AmisLVNKJlGbRs3 2qIhO8PAGP8Kli/Dff7NZ5bfBZob2nbZ6CEG0Qv/UHHcNzrBdzMA0gdGsuOpwjSW oJjqtgu4jfXlNAkPZKIvUk1wYUhxAN/2AZ2kfriLOJCeT7QOxYawKXWTjhmcj/QM mZfJTDhebDtqR/WLgjlISQ/pzyKl+lUl0sV0+6FNM1pycPQSrv1yKAD9dMaDpUFA y0WU6MJ+nig6uIYVZ2W9DKGu7qJp6Ghdi+IcmqHFqu/XEnw2LuG0ox2D5NdEkbc9 Ulp4KQOLuU9gSwQKNrPh =YRky -----END PGP SIGNATURE----- _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
<overly trimmed>
On 02/25/2015 01:56 PM, Ashley M. Kirchner wrote:
Ok, so some of this now works, but I'm still having problems. With the bootif option, the system now correctly configures and uses the same interface to get its kickstart file. However, when the system is done and boots up, the interfaces are still messed up. So this is what I have in the kickstart file:
What version of CentOS 6 is this?
In the PXE config file I have:
IPAPPEND 2 APPEND ks=http://192.168.x.x/ks/portico.ks initrd=centos/x86_64/initrd.img ramdisk_size=100000 ksdevice=bootif
As soon as I *remove* the additional ethernet card, the system will boot up with the ports configured correctly (port 1 = eth0, port 2 = eth1). So why is it that as soon as there is an additional one, all things go to hell? Why must the boot process shuffle them? More importantly, how do I prevent this so that the system comes up properly after a kickstart install?
The reason I ask the version, is this is exactly the sort of thing that biosdevname is designed to solve. With biosdevname, you get devices like 'em1, em2, p6p1', which aren't as friendly as 'eth0' but also keep names sane and avoid the hair-tearing issues you're experiencing currently. You don't appear to be adding anything via your append line that would disable biosdevname, so I must assume you're using a much older 6 base install.
Version 6.6 ...
On Wed, Feb 25, 2015 at 1:17 PM, Jim Perrin jperrin@centos.org wrote:
<overly trimmed>
On 02/25/2015 01:56 PM, Ashley M. Kirchner wrote:
Ok, so some of this now works, but I'm still having problems. With the bootif option, the system now correctly configures and uses the same interface to get its kickstart file. However, when the system is done and boots up, the interfaces are still messed up. So this is what I have in
the
kickstart file:
What version of CentOS 6 is this?
In the PXE config file I have:
IPAPPEND 2 APPEND ks=http://192.168.x.x/ks/portico.ks
initrd=centos/x86_64/initrd.img
ramdisk_size=100000 ksdevice=bootif
As soon as I *remove* the additional ethernet card, the system will boot
up
with the ports configured correctly (port 1 = eth0, port 2 = eth1). So
why
is it that as soon as there is an additional one, all things go to hell? Why must the boot process shuffle them? More importantly, how do I
prevent
this so that the system comes up properly after a kickstart install?
The reason I ask the version, is this is exactly the sort of thing that biosdevname is designed to solve. With biosdevname, you get devices like 'em1, em2, p6p1', which aren't as friendly as 'eth0' but also keep names sane and avoid the hair-tearing issues you're experiencing currently. You don't appear to be adding anything via your append line that would disable biosdevname, so I must assume you're using a much older 6 base install.
-- Jim Perrin The CentOS Project | http://www.centos.org twitter: @BitIntegrity | GPG Key: FA09AD77 _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Starting back in RHEL/Cent 5 I found that the only way to make sure your interface enumeration was consistent after install with what you had during install was to create a udev rules file using the mac addresses as the key. It is easy to run a short script in postinstall to create it based on how anaconda has seen them.
In order for this to work on Cent 6 you have to set biosdevname=0 on the kernel boot for the installed system.
PXE boot options:
label c6inst-sda kernel /linux-boot/cent6-x64/vmlinuz append initrd=/linux-boot/cent6-x64/initrd.img ksdevice=bootif ip=dhcp ks=http://xx.xx.xx.xx/install/linux/ks/basic-cent6-sda.cfg ipappend 2
In kickstart:
BOOTOPTS="biosdevname=0"
Also in kickstart I do not specify the config for ANY network interfaces. I let anaconda pull in only the config for the boot interface from PXE. I manually configure everything else. The only thing I do to non-boot interfaces is set the DHCP and ONBOOT to no.
On Wed, 25 Feb 2015 14:21:18 -0600, Ashley M. Kirchner ashley@pcraft.com wrote:
Version 6.6 ...
On Wed, Feb 25, 2015 at 1:17 PM, Jim Perrin jperrin@centos.org wrote:
<overly trimmed>
On 02/25/2015 01:56 PM, Ashley M. Kirchner wrote:
Ok, so some of this now works, but I'm still having problems. With the bootif option, the system now correctly configures and uses the same interface to get its kickstart file. However, when the system is done
and
boots up, the interfaces are still messed up. So this is what I have
in the
kickstart file:
What version of CentOS 6 is this?
In the PXE config file I have:
IPAPPEND 2 APPEND ks=http://192.168.x.x/ks/portico.ks
initrd=centos/x86_64/initrd.img
ramdisk_size=100000 ksdevice=bootif
As soon as I *remove* the additional ethernet card, the system will
boot up
with the ports configured correctly (port 1 = eth0, port 2 = eth1). So
why
is it that as soon as there is an additional one, all things go to
hell?
Why must the boot process shuffle them? More importantly, how do I
prevent
this so that the system comes up properly after a kickstart install?
The reason I ask the version, is this is exactly the sort of thing that biosdevname is designed to solve. With biosdevname, you get devices like 'em1, em2, p6p1', which aren't as friendly as 'eth0' but also keep names sane and avoid the hair-tearing issues you're experiencing currently. You don't appear to be adding anything via your append line that would disable biosdevname, so I must assume you're using a much older 6 base install.
In my experience biosdevname creates just as many problems as it solves. Dell can keep it.
-- Jim Perrin The CentOS Project | http://www.centos.org twitter: @BitIntegrity | GPG Key: FA09AD77 _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Thanks for that Jason but it didn't solve the problem. The system is still coming up with the interfaces shuffled. It seems to *always* want to use the added ethernet card as eth0.
On Wed, Feb 25, 2015 at 1:37 PM, Jason Warr jason@warr.net wrote:
Starting back in RHEL/Cent 5 I found that the only way to make sure your interface enumeration was consistent after install with what you had during install was to create a udev rules file using the mac addresses as the key. It is easy to run a short script in postinstall to create it based on how anaconda has seen them.
In order for this to work on Cent 6 you have to set biosdevname=0 on the kernel boot for the installed system.
PXE boot options:
label c6inst-sda kernel /linux-boot/cent6-x64/vmlinuz append initrd=/linux-boot/cent6-x64/initrd.img ksdevice=bootif ip=dhcp ks=http://xx.xx.xx.xx/install/linux/ks/basic-cent6-sda.cfg ipappend 2
In kickstart:
BOOTOPTS="biosdevname=0"
Also in kickstart I do not specify the config for ANY network interfaces. I let anaconda pull in only the config for the boot interface from PXE. I manually configure everything else. The only thing I do to non-boot interfaces is set the DHCP and ONBOOT to no.
On Wed, 25 Feb 2015 14:21:18 -0600, Ashley M. Kirchner ashley@pcraft.com wrote:
Version 6.6 ...
On Wed, Feb 25, 2015 at 1:17 PM, Jim Perrin jperrin@centos.org wrote:
<overly trimmed> > > On 02/25/2015 01:56 PM, Ashley M. Kirchner wrote: > > Ok, so some of this now works, but I'm still having problems. With the > > bootif option, the system now correctly configures and uses the same > > interface to get its kickstart file. However, when the system is done > and > > boots up, the interfaces are still messed up. So this is what I have in > the > > kickstart file: > > What version of CentOS 6 is this? > > > In the PXE config file I have: > > > > IPAPPEND 2 > > APPEND ks=http://192.168.x.x/ks/portico.ks > initrd=centos/x86_64/initrd.img > > ramdisk_size=100000 ksdevice=bootif > > > As soon as I *remove* the additional ethernet card, the system will > boot > up > > with the ports configured correctly (port 1 = eth0, port 2 = eth1). So > why > > is it that as soon as there is an additional one, all things go to > hell? > > Why must the boot process shuffle them? More importantly, how do I > prevent > > this so that the system comes up properly after a kickstart install? > > > > The reason I ask the version, is this is exactly the sort of thing that > biosdevname is designed to solve. With biosdevname, you get devices like > 'em1, em2, p6p1', which aren't as friendly as 'eth0' but also keep names > sane and avoid the hair-tearing issues you're experiencing currently. > You don't appear to be adding anything via your append line that would > disable biosdevname, so I must assume you're using a much older 6 base > install. > >
In my experience biosdevname creates just as many problems as it solves. Dell can keep it.
-- Jim Perrin The CentOS Project | http://www.centos.org twitter: @BitIntegrity | GPG Key: FA09AD77 _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Here is my script for post install if you want to try it.
In order for the shuffling to not occur you do need to create the udev rules file somehow. I am not sure how mangled this will be in email but it is worth a try. It should run OK with nothing else. I have a better version in the works but the enhancements are mainly useful for Fedora 19-21.
I did forget to say I also block NetworkManager from the interfaces.
############################
#!/bin/bash ## BIND MAC address to proper interfaces so they stay persistent ## I want them to stay as they were in kickstart
## This can cause issues with VLAN interfaces for both bond dev's and base eth dev's. ## The bond one was solved by adding in the "KERNEL="eth?*" as that will only apply to physical ## Devices. Once we have VLAN's on a real device instead of just on BOND's this then applies ## to the hardware devices as well. The core issue is that the MAC address is inherited ## by all of the children devices and thus the UDEV rule has to be crafted to only apply ## to the base physical device. ## This one was solved via adding DRIVERS=="?*" as the VLAN int's wont have one
echo "[KICKSTART] Binding eth interfaces to the expected MAC address in UDEV" echo "## Created by Kickstart to keep network interfaces in an expected order" > \ /etc/udev/rules.d/70-persistent-net.rules echo "" >> \ /etc/udev/rules.d/70-persistent-net.rules
cd /sys/class/net/ for NETDEV in $(ls | grep eth | sort) do ## Create a UDEV rule for each eth interface echo "## ${NETDEV} interface" >> \ /etc/udev/rules.d/70-persistent-net.rules
## We throw this one in here as it can contain some useful information echo "## $(dmesg | grep ${NETDEV} | grep -i -v -e "console" -e "Command line" | head -1)" >> \ /etc/udev/rules.d/70-persistent-net.rules
echo -n "SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", "
\
/etc/udev/rules.d/70-persistent-net.rules echo -n "ATTR{address}=="$(cat ${NETDEV}/address)", " >> \ /etc/udev/rules.d/70-persistent-net.rules echo -n "ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth?*", " >> \ /etc/udev/rules.d/70-persistent-net.rules echo -e "NAME="${NETDEV}"\n" >> \ /etc/udev/rules.d/70-persistent-net.rules
## Make a log of the devices present during install echo -e "${NETDEV} $(cat ${NETDEV}/address)\n" >> /root/ksnet-devices
## Also remove the HWADDR line from all of the net config files grep -v -e NAME -e HWADDR -e NM_CONTROLLED \ /etc/sysconfig/network-scripts/ifcfg-${NETDEV} | sed 's/"//g' \ > /etc/sysconfig/network-scripts/ifcfg-${NETDEV}-tmp echo "NM_CONTROLLED=no" >> /etc/sysconfig/network-scripts/ifcfg-${NETDEV}-tmp /usr/bin/perl -p -i -e 's/dhcp/none/' /etc/sysconfig/network-scripts/ifcfg-${NETDEV}-tmp mv -f /etc/sysconfig/network-scripts/ifcfg-${NETDEV}-tmp \ /etc/sysconfig/network-scripts/ifcfg-${NETDEV} done
###########################
On Wed, 25 Feb 2015 14:53:40 -0600, Ashley M. Kirchner ashley@pcraft.com wrote:
Thanks for that Jason but it didn't solve the problem. The system is still coming up with the interfaces shuffled. It seems to *always* want to use the added ethernet card as eth0.
On Wed, Feb 25, 2015 at 1:37 PM, Jason Warr jason@warr.net wrote:
Starting back in RHEL/Cent 5 I found that the only way to make sure your interface enumeration was consistent after install with what you had during install was to create a udev rules file using the mac addresses as the key. It is easy to run a short script in postinstall to create it based on how anaconda has seen them.
In order for this to work on Cent 6 you have to set biosdevname=0 on the kernel boot for the installed system.
PXE boot options:
label c6inst-sda kernel /linux-boot/cent6-x64/vmlinuz append initrd=/linux-boot/cent6-x64/initrd.img ksdevice=bootif ip=dhcp ks=http://xx.xx.xx.xx/install/linux/ks/basic-cent6-sda.cfg ipappend 2
In kickstart:
BOOTOPTS="biosdevname=0"
Also in kickstart I do not specify the config for ANY network interfaces. I let anaconda pull in only the config for the boot interface from PXE. I manually configure everything else. The only thing I do to non-boot interfaces is set the DHCP and ONBOOT to no.
On Wed, 25 Feb 2015 14:21:18 -0600, Ashley M. Kirchner ashley@pcraft.com wrote:
Version 6.6 ...
On Wed, Feb 25, 2015 at 1:17 PM, Jim Perrin jperrin@centos.org wrote:
<overly trimmed> > > On 02/25/2015 01:56 PM, Ashley M. Kirchner wrote: > > Ok, so some of this now works, but I'm still having problems. With > the > > bootif option, the system now correctly configures and uses the same > > interface to get its kickstart file. However, when the system is > done > and > > boots up, the interfaces are still messed up. So this is what I > have in > the > > kickstart file: > > What version of CentOS 6 is this? > > > In the PXE config file I have: > > > > IPAPPEND 2 > > APPEND ks=http://192.168.x.x/ks/portico.ks > initrd=centos/x86_64/initrd.img > > ramdisk_size=100000 ksdevice=bootif > > > As soon as I *remove* the additional ethernet card, the system will > boot > up > > with the ports configured correctly (port 1 = eth0, port 2 = eth1). > So > why > > is it that as soon as there is an additional one, all things go to > hell? > > Why must the boot process shuffle them? More importantly, how do I > prevent > > this so that the system comes up properly after a kickstart install? > > > > The reason I ask the version, is this is exactly the sort of thing > that > biosdevname is designed to solve. With biosdevname, you get devices > like > 'em1, em2, p6p1', which aren't as friendly as 'eth0' but also keep > names > sane and avoid the hair-tearing issues you're experiencing currently. > You don't appear to be adding anything via your append line that would > disable biosdevname, so I must assume you're using a much older 6 base > install. > >
In my experience biosdevname creates just as many problems as it solves. Dell can keep it.
-- Jim Perrin The CentOS Project | http://www.centos.org twitter: @BitIntegrity | GPG Key: FA09AD77 _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Ok, when I run that, it creates a now "custom" 70-persistent-net.rules, but the interfaces are still out of order, with the added one listed first, and the built-ins 2nd and 3rd.
On Wed, Feb 25, 2015 at 2:00 PM, Jason Warr jason@warr.net wrote:
Here is my script for post install if you want to try it.
In order for the shuffling to not occur you do need to create the udev rules file somehow. I am not sure how mangled this will be in email but it is worth a try. It should run OK with nothing else. I have a better version in the works but the enhancements are mainly useful for Fedora 19-21.
I did forget to say I also block NetworkManager from the interfaces.
############################
#!/bin/bash ## BIND MAC address to proper interfaces so they stay persistent ## I want them to stay as they were in kickstart
## This can cause issues with VLAN interfaces for both bond dev's and base eth dev's. ## The bond one was solved by adding in the "KERNEL="eth?*" as that will only apply to physical ## Devices. Once we have VLAN's on a real device instead of just on BOND's this then applies ## to the hardware devices as well. The core issue is that the MAC address is inherited ## by all of the children devices and thus the UDEV rule has to be crafted to only apply ## to the base physical device. ## This one was solved via adding DRIVERS=="?*" as the VLAN int's wont have one
echo "[KICKSTART] Binding eth interfaces to the expected MAC addressin UDEV" echo "## Created by Kickstart to keep network interfaces in an expected order" > \ /etc/udev/rules.d/70-persistent-net.rules echo "" >> \ /etc/udev/rules.d/70-persistent-net.rules
cd /sys/class/net/ for NETDEV in $(ls | grep eth | sort) do ## Create a UDEV rule for each eth interface echo "## ${NETDEV} interface" >> \ /etc/udev/rules.d/70-persistent-net.rules ## We throw this one in here as it can contain some usefulinformation echo "## $(dmesg | grep ${NETDEV} | grep -i -v -e "console" -e "Command line" | head -1)" >> \ /etc/udev/rules.d/70-persistent-net.rules
echo -n "SUBSYSTEM==\"net\", ACTION==\"add\", DRIVERS==\"?*\", "\
/etc/udev/rules.d/70-persistent-net.rulesecho -n "ATTR{address}==\"$(cat ${NETDEV}/address)\", " >> \ /etc/udev/rules.d/70-persistent-net.rules echo -n "ATTR{dev_id}==\"0x0\", ATTR{type}==\"1\",KERNEL=="eth?*", " >> \ /etc/udev/rules.d/70-persistent-net.rules echo -e "NAME="${NETDEV}"\n" >> \ /etc/udev/rules.d/70-persistent-net.rules
## Make a log of the devices present during install echo -e "${NETDEV} $(cat ${NETDEV}/address)\n" >>/root/ksnet-devices
## Also remove the HWADDR line from all of the net config files grep -v -e NAME -e HWADDR -e NM_CONTROLLED \ /etc/sysconfig/network-scripts/ifcfg-${NETDEV} | sed's/"//g' \ > /etc/sysconfig/network-scripts/ifcfg-${NETDEV}-tmp echo "NM_CONTROLLED=no" >> /etc/sysconfig/network- scripts/ifcfg-${NETDEV}-tmp /usr/bin/perl -p -i -e 's/dhcp/none/' /etc/sysconfig/network- scripts/ifcfg-${NETDEV}-tmp mv -f /etc/sysconfig/network-scripts/ifcfg-${NETDEV}-tmp \ /etc/sysconfig/network-scripts/ifcfg-${NETDEV} done
###########################
On Wed, 25 Feb 2015 14:53:40 -0600, Ashley M. Kirchner ashley@pcraft.com wrote:
Thanks for that Jason but it didn't solve the problem. The system is still
coming up with the interfaces shuffled. It seems to *always* want to use the added ethernet card as eth0.
On Wed, Feb 25, 2015 at 1:37 PM, Jason Warr jason@warr.net wrote:
Starting back in RHEL/Cent 5 I found that the only way to make sure your
interface enumeration was consistent after install with what you had during install was to create a udev rules file using the mac addresses as the key. It is easy to run a short script in postinstall to create it based on how anaconda has seen them.
In order for this to work on Cent 6 you have to set biosdevname=0 on the kernel boot for the installed system.
PXE boot options:
label c6inst-sda kernel /linux-boot/cent6-x64/vmlinuz append initrd=/linux-boot/cent6-x64/initrd.img ksdevice=bootif ip=dhcp ks=http://xx.xx.xx.xx/install/linux/ks/basic-cent6-sda.cfg ipappend 2
In kickstart:
BOOTOPTS="biosdevname=0"
Also in kickstart I do not specify the config for ANY network interfaces. I let anaconda pull in only the config for the boot interface from PXE. I manually configure everything else. The only thing I do to non-boot interfaces is set the DHCP and ONBOOT to no.
On Wed, 25 Feb 2015 14:21:18 -0600, Ashley M. Kirchner < ashley@pcraft.com> wrote:
Version 6.6 ...
On Wed, Feb 25, 2015 at 1:17 PM, Jim Perrin jperrin@centos.org wrote:
<overly trimmed>
On 02/25/2015 01:56 PM, Ashley M. Kirchner wrote:
Ok, so some of this now works, but I'm still having problems. With
the
bootif option, the system now correctly configures and uses the same interface to get its kickstart file. However, when the system is done
and
boots up, the interfaces are still messed up. So this is what I have
in the
kickstart file:
What version of CentOS 6 is this?
In the PXE config file I have:
IPAPPEND 2 APPEND ks=http://192.168.x.x/ks/portico.ks
initrd=centos/x86_64/initrd.img
ramdisk_size=100000 ksdevice=bootif
As soon as I *remove* the additional ethernet card, the system will
boot up
with the ports configured correctly (port 1 = eth0, port 2 = eth1).
So why
is it that as soon as there is an additional one, all things go to
hell?
Why must the boot process shuffle them? More importantly, how do I
prevent
this so that the system comes up properly after a kickstart install?
The reason I ask the version, is this is exactly the sort of thing that biosdevname is designed to solve. With biosdevname, you get devices like 'em1, em2, p6p1', which aren't as friendly as 'eth0' but also keep names sane and avoid the hair-tearing issues you're experiencing currently. You don't appear to be adding anything via your append line that would disable biosdevname, so I must assume you're using a much older 6 base install.
In my experience biosdevname creates just as many problems as it
solves. Dell can keep it.
--
Jim Perrin The CentOS Project | http://www.centos.org twitter: @BitIntegrity | GPG Key: FA09AD77 _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Define out of order in this case just so I know for sure what you mean.
What my solution does, or at least does reliably in my case, is make sure the interfaces are in the same order once installed as the install kernel saw them. It won't re-order them to be sequential based on bus, mac or driver. I am working on that but it will also include naming the devices based on the module name, similar to how FreeBSD and Solaris do it.
Just to get an idea of what might be going on can you run "dmesg | grep eth" so I can see some of what udev is doing?
On Wed, 25 Feb 2015 16:28:31 -0600, Ashley M. Kirchner ashley@pcraft.com wrote:
Ok, when I run that, it creates a now "custom" 70-persistent-net.rules, but the interfaces are still out of order, with the added one listed first, and the built-ins 2nd and 3rd.
On Wed, Feb 25, 2015 at 2:00 PM, Jason Warr jason@warr.net wrote:
Here is my script for post install if you want to try it.
In order for the shuffling to not occur you do need to create the udev rules file somehow. I am not sure how mangled this will be in email but it is worth a try. It should run OK with nothing else. I have a better version in the works but the enhancements are mainly useful for Fedora 19-21.
I did forget to say I also block NetworkManager from the interfaces.
############################
#!/bin/bash ## BIND MAC address to proper interfaces so they stay persistent ## I want them to stay as they were in kickstart
## This can cause issues with VLAN interfaces for both bond dev's and base eth dev's. ## The bond one was solved by adding in the "KERNEL="eth?*" as that will only apply to physical ## Devices. Once we have VLAN's on a real device instead of just on BOND's this then applies ## to the hardware devices as well. The core issue is that the MAC address is inherited ## by all of the children devices and thus the UDEV rule has to be crafted to only apply ## to the base physical device. ## This one was solved via adding DRIVERS=="?*" as the VLAN int's wont have one
echo "[KICKSTART] Binding eth interfaces to the expected MAC addressin UDEV" echo "## Created by Kickstart to keep network interfaces in an expected order" > \ /etc/udev/rules.d/70-persistent-net.rules echo "" >> \ /etc/udev/rules.d/70-persistent-net.rules
cd /sys/class/net/ for NETDEV in $(ls | grep eth | sort) do ## Create a UDEV rule for each eth interface echo "## ${NETDEV} interface" >> \ /etc/udev/rules.d/70-persistent-net.rules ## We throw this one in here as it can contain some usefulinformation echo "## $(dmesg | grep ${NETDEV} | grep -i -v -e "console" -e "Command line" | head -1)" >> \ /etc/udev/rules.d/70-persistent-net.rules
echo -n "SUBSYSTEM==\"net\", ACTION==\"add\", DRIVERS==\"?*\", "\
/etc/udev/rules.d/70-persistent-net.rulesecho -n "ATTR{address}==\"$(cat ${NETDEV}/address)\", " >> \ /etc/udev/rules.d/70-persistent-net.rules echo -n "ATTR{dev_id}==\"0x0\", ATTR{type}==\"1\",KERNEL=="eth?*", " >> \ /etc/udev/rules.d/70-persistent-net.rules echo -e "NAME="${NETDEV}"\n" >> \ /etc/udev/rules.d/70-persistent-net.rules
## Make a log of the devices present during install echo -e "${NETDEV} $(cat ${NETDEV}/address)\n" >>/root/ksnet-devices
## Also remove the HWADDR line from all of the net config files grep -v -e NAME -e HWADDR -e NM_CONTROLLED \ /etc/sysconfig/network-scripts/ifcfg-${NETDEV} | sed's/"//g' \ > /etc/sysconfig/network-scripts/ifcfg-${NETDEV}-tmp echo "NM_CONTROLLED=no" >> /etc/sysconfig/network- scripts/ifcfg-${NETDEV}-tmp /usr/bin/perl -p -i -e 's/dhcp/none/' /etc/sysconfig/network- scripts/ifcfg-${NETDEV}-tmp mv -f /etc/sysconfig/network-scripts/ifcfg-${NETDEV}-tmp \ /etc/sysconfig/network-scripts/ifcfg-${NETDEV} done
###########################
On Wed, 25 Feb 2015 14:53:40 -0600, Ashley M. Kirchner ashley@pcraft.com wrote:
Thanks for that Jason but it didn't solve the problem. The system is still
coming up with the interfaces shuffled. It seems to *always* want to use the added ethernet card as eth0.
On Wed, Feb 25, 2015 at 1:37 PM, Jason Warr jason@warr.net wrote:
Starting back in RHEL/Cent 5 I found that the only way to make sure your
interface enumeration was consistent after install with what you had during install was to create a udev rules file using the mac addresses as the key. It is easy to run a short script in postinstall to create it based on how anaconda has seen them.
In order for this to work on Cent 6 you have to set biosdevname=0 on the kernel boot for the installed system.
PXE boot options:
label c6inst-sda kernel /linux-boot/cent6-x64/vmlinuz append initrd=/linux-boot/cent6-x64/initrd.img ksdevice=bootif ip=dhcp ks=http://xx.xx.xx.xx/install/linux/ks/basic-cent6-sda.cfg ipappend 2
In kickstart:
BOOTOPTS="biosdevname=0"
Also in kickstart I do not specify the config for ANY network interfaces. I let anaconda pull in only the config for the boot interface from PXE. I manually configure everything else. The only thing I do to non-boot interfaces is set the DHCP and ONBOOT to no.
On Wed, 25 Feb 2015 14:21:18 -0600, Ashley M. Kirchner < ashley@pcraft.com> wrote:
Version 6.6 ...
On Wed, Feb 25, 2015 at 1:17 PM, Jim Perrin jperrin@centos.org wrote:
<overly trimmed>
On 02/25/2015 01:56 PM, Ashley M. Kirchner wrote: > Ok, so some of this now works, but I'm still having problems. With the > bootif option, the system now correctly configures and uses the same > interface to get its kickstart file. However, when the system is done and > boots up, the interfaces are still messed up. So this is what I have in the > kickstart file:
What version of CentOS 6 is this?
> In the PXE config file I have: > > IPAPPEND 2 > APPEND ks=http://192.168.x.x/ks/portico.ks initrd=centos/x86_64/initrd.img > ramdisk_size=100000 ksdevice=bootif
> As soon as I *remove* the additional ethernet card, the system will boot up > with the ports configured correctly (port 1 = eth0, port 2 = eth1). So why > is it that as soon as there is an additional one, all things go to hell? > Why must the boot process shuffle them? More importantly, how do I prevent > this so that the system comes up properly after a kickstart install? >
The reason I ask the version, is this is exactly the sort of thing that biosdevname is designed to solve. With biosdevname, you get devices like 'em1, em2, p6p1', which aren't as friendly as 'eth0' but also keep names sane and avoid the hair-tearing issues you're experiencing currently. You don't appear to be adding anything via your append line that would disable biosdevname, so I must assume you're using a much older 6 base install.
In my experience biosdevname creates just as many problems as it
solves. Dell can keep it.
--
Jim Perrin The CentOS Project | http://www.centos.org twitter: @BitIntegrity | GPG Key: FA09AD77 _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Out of order meaning, it puts the additional ethernet card as eth0, with the built-in ports as eth1 and eth2 respectively. WITHOUT the additional card installed, it puts the built-in ports as eth0 and eth1, which is what I want it to do. The additional card should be eth2, at least that's what I want it to do.
Looking at persistent-net.rules, it always puts the additional card first, both without your script as well as with. I need it to be last. The system's built-ins should always be eth0 and eth1 respectively.
And dmesg confirms it as well, it identifies the added card first (and assigns it eth0), then identifies the built-in ports. I'd grab the actual output except I need to manually reconfigure the interfaces so I can actually get ON the machine. Right now I'm just looking at its console.
On Wed, Feb 25, 2015 at 3:37 PM, Jason Warr jason@warr.net wrote:
Define out of order in this case just so I know for sure what you mean.
What my solution does, or at least does reliably in my case, is make sure the interfaces are in the same order once installed as the install kernel saw them. It won't re-order them to be sequential based on bus, mac or driver. I am working on that but it will also include naming the devices based on the module name, similar to how FreeBSD and Solaris do it.
Just to get an idea of what might be going on can you run "dmesg | grep eth" so I can see some of what udev is doing?
On Wed, 25 Feb 2015 16:28:31 -0600, Ashley M. Kirchner ashley@pcraft.com wrote:
Ok, when I run that, it creates a now "custom" 70-persistent-net.rules,
but the interfaces are still out of order, with the added one listed first, and the built-ins 2nd and 3rd.
On Wed, Feb 25, 2015 at 2:00 PM, Jason Warr jason@warr.net wrote:
Here is my script for post install if you want to try it.
In order for the shuffling to not occur you do need to create the udev rules file somehow. I am not sure how mangled this will be in email but it is worth a try. It should run OK with nothing else. I have a better version in the works but the enhancements are mainly useful for Fedora 19-21.
I did forget to say I also block NetworkManager from the interfaces.
############################
#!/bin/bash ## BIND MAC address to proper interfaces so they stay persistent ## I want them to stay as they were in kickstart
## This can cause issues with VLAN interfaces for both bond dev's and base eth dev's. ## The bond one was solved by adding in the "KERNEL="eth?*" as that will only apply to physical ## Devices. Once we have VLAN's on a real device instead of just on BOND's this then applies ## to the hardware devices as well. The core issue is that the MAC address is inherited ## by all of the children devices and thus the UDEV rule has to be crafted to only apply ## to the base physical device. ## This one was solved via adding DRIVERS=="?*" as the VLAN int's wont have one
echo "[KICKSTART] Binding eth interfaces to the expected MAC addressin UDEV" echo "## Created by Kickstart to keep network interfaces in an expected order" > \ /etc/udev/rules.d/70-persistent-net.rules echo "" >> \ /etc/udev/rules.d/70-persistent-net.rules
cd /sys/class/net/ for NETDEV in $(ls | grep eth | sort) do ## Create a UDEV rule for each eth interface echo "## ${NETDEV} interface" >> \ /etc/udev/rules.d/70-persistent-net.rules ## We throw this one in here as it can contain some usefulinformation echo "## $(dmesg | grep ${NETDEV} | grep -i -v -e "console" -e "Command line" | head -1)" >> \ /etc/udev/rules.d/70-persistent-net.rules
echo -n "SUBSYSTEM==\"net\", ACTION==\"add\", DRIVERS==\"?*\", "\
/etc/udev/rules.d/70-persistent-net.rulesecho -n "ATTR{address}==\"$(cat ${NETDEV}/address)\", " >> \ /etc/udev/rules.d/70-persistent-net.rules echo -n "ATTR{dev_id}==\"0x0\", ATTR{type}==\"1\",KERNEL=="eth?*", " >> \ /etc/udev/rules.d/70-persistent-net.rules echo -e "NAME="${NETDEV}"\n" >> \ /etc/udev/rules.d/70-persistent-net.rules
## Make a log of the devices present during install echo -e "${NETDEV} $(cat ${NETDEV}/address)\n" >>/root/ksnet-devices
## Also remove the HWADDR line from all of the net config files grep -v -e NAME -e HWADDR -e NM_CONTROLLED \ /etc/sysconfig/network-scripts/ifcfg-${NETDEV} | sed's/"//g' \ > /etc/sysconfig/network-scripts/ifcfg-${NETDEV}-tmp echo "NM_CONTROLLED=no" >> /etc/sysconfig/network- scripts/ifcfg-${NETDEV}-tmp /usr/bin/perl -p -i -e 's/dhcp/none/' /etc/sysconfig/network- scripts/ifcfg-${NETDEV}-tmp mv -f /etc/sysconfig/network-scripts/ifcfg-${NETDEV}-tmp \ /etc/sysconfig/network-scripts/ifcfg-${NETDEV} done
###########################
On Wed, 25 Feb 2015 14:53:40 -0600, Ashley M. Kirchner < ashley@pcraft.com> wrote:
Thanks for that Jason but it didn't solve the problem. The system is still
coming up with the interfaces shuffled. It seems to *always* want to use the added ethernet card as eth0.
On Wed, Feb 25, 2015 at 1:37 PM, Jason Warr jason@warr.net wrote:
Starting back in RHEL/Cent 5 I found that the only way to make sure your
interface enumeration was consistent after install with what you had during install was to create a udev rules file using the mac addresses as the key. It is easy to run a short script in postinstall to create it based on how anaconda has seen them.
In order for this to work on Cent 6 you have to set biosdevname=0 on the kernel boot for the installed system.
PXE boot options:
label c6inst-sda kernel /linux-boot/cent6-x64/vmlinuz append initrd=/linux-boot/cent6-x64/initrd.img ksdevice=bootif ip=dhcp ks=http://xx.xx.xx.xx/install/linux/ks/basic-cent6-sda.cfg ipappend 2
In kickstart:
BOOTOPTS="biosdevname=0"
Also in kickstart I do not specify the config for ANY network interfaces. I let anaconda pull in only the config for the boot interface from PXE. I manually configure everything else. The only thing I do to non-boot interfaces is set the DHCP and ONBOOT to no.
On Wed, 25 Feb 2015 14:21:18 -0600, Ashley M. Kirchner < ashley@pcraft.com> wrote:
Version 6.6 ...
On Wed, Feb 25, 2015 at 1:17 PM, Jim Perrin jperrin@centos.org wrote:
<overly trimmed>
> On 02/25/2015 01:56 PM, Ashley M. Kirchner wrote: > > Ok, so some of this now works, but I'm still having problems. With > the > > bootif option, the system now correctly configures and uses the > same > > interface to get its kickstart file. However, when the system is > done > and > > boots up, the interfaces are still messed up. So this is what I > have > in > the > > kickstart file: > > What version of CentOS 6 is this? > > > In the PXE config file I have: > > > > IPAPPEND 2 > > APPEND ks=http://192.168.x.x/ks/portico.ks > initrd=centos/x86_64/initrd.img > > ramdisk_size=100000 ksdevice=bootif > > > As soon as I *remove* the additional ethernet card, the system will > boot > up > > with the ports configured correctly (port 1 = eth0, port 2 = eth1). > So > why > > is it that as soon as there is an additional one, all things go to > hell? > > Why must the boot process shuffle them? More importantly, how do I > prevent > > this so that the system comes up properly after a kickstart > install? > > > > The reason I ask the version, is this is exactly the sort of thing > that > biosdevname is designed to solve. With biosdevname, you get devices > like > 'em1, em2, p6p1', which aren't as friendly as 'eth0' but also keep > names > sane and avoid the hair-tearing issues you're experiencing currently. > You don't appear to be adding anything via your append line that > would > disable biosdevname, so I must assume you're using a much older 6 > base > install. > > > In my experience biosdevname creates just as many problems as it > solves.
Dell can keep it.
--
Jim Perrin > The CentOS Project | http://www.centos.org > twitter: @BitIntegrity | GPG Key: FA09AD77 > _______________________________________________ > CentOS mailing list > CentOS@centos.org > http://lists.centos.org/mailman/listinfo/centos > > > _______________________________________________ > > CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Ashley M. Kirchner wrote:
Out of order meaning, it puts the additional ethernet card as eth0, with the built-in ports as eth1 and eth2 respectively. WITHOUT the additional card installed, it puts the built-in ports as eth0 and eth1, which is what I want it to do. The additional card should be eth2, at least that's what I want it to do.
<snip> Now that you have a 70-persistant-net.rules, what happens if you edit it, and name the interfaces in the correct order, then reboot?
mark
On Wed, 25 Feb 2015 16:45:04 -0600, Ashley M. Kirchner ashley@pcraft.com wrote:
Out of order meaning, it puts the additional ethernet card as eth0, with the built-in ports as eth1 and eth2 respectively. WITHOUT the additional card >installed, it puts the built-in ports as eth0 and eth1, which is what I want it to do. The additional card should be eth2, at least that's what I want it to >do.
So if you really need them to be in that order your best bet may be to blacklist the kernel module for the add in card so it does not get enumerated during install. If you do that and create the udev rules file as I do then on first boot udev should enforce your rules file and then enumerate the add in card as eth2.
Add "rdblacklist=MODULE_NAME" to your append line in pxelinux.conf file.
This should get you into a usable state post-install so you can setup additional interfaces.
Looking at persistent-net.rules, it always puts the additional card first, both without your script as well as with. I need it to be last. The system's built->ins should always be eth0 and eth1 respectively.
It looks like the add in card is in a bus slot that is getting enumerated, module loaded before the on-board interfaces.
And dmesg confirms it as well, it identifies the added card first (and assigns it eth0), then identifies the built-in ports. I'd grab the actual output except >I need to manually reconfigure the interfaces so I can actually get ON the machine. Right now I'm just looking at its console
On Wed, Feb 25, 2015 at 3:37 PM, Jason Warr jason@warr.net wrote:
Define out of order in this case just so I know for sure what you mean.
What my solution does, or at least does reliably in my case, is make sure the interfaces are in the same order once installed as the install kernel saw >>them. It won't re-order them to be sequential based on bus, mac or driver. I am working on that but it will also include naming the devices based on >>the module name, similar to how FreeBSD and Solaris do it.
Just to get an idea of what might be going on can you run "dmesg | grep eth" so I can see some of what udev is doing?
On Wed, 25 Feb 2015 16:28:31 -0600, Ashley M. Kirchner ashley@pcraft.com wrote:
Ok, when I run that, it creates a now "custom" 70-persistent-net.rules, but the interfaces are still out of order, with the added one listed first, and the built-ins 2nd and 3rd.
On Wed, Feb 25, 2015 at 2:00 PM, Jason Warr jason@warr.net wrote:
Here is my script for post install if you want to try it.
In order for the shuffling to not occur you do need to create the udev rules file somehow. I am not sure how mangled this will be in email but it is worth a try. It should run OK with nothing else. I have a better version in the works but the enhancements are mainly useful for Fedora 19-21.
I did forget to say I also block NetworkManager from the interfaces.
############################
#!/bin/bash ## BIND MAC address to proper interfaces so they stay persistent ## I want them to stay as they were in kickstart
## This can cause issues with VLAN interfaces for both bond dev's and base eth dev's. ## The bond one was solved by adding in the "KERNEL="eth?*" as that will only apply to physical ## Devices. Once we have VLAN's on a real device instead of just on BOND's this then applies ## to the hardware devices as well. The core issue is that the MAC address is inherited ## by all of the children devices and thus the UDEV rule has to be crafted to only apply ## to the base physical device. ## This one was solved via adding DRIVERS=="?*" as the VLAN int's wont have one
echo "[KICKSTART] Binding eth interfaces to the expected MAC address in UDEV" echo "## Created by Kickstart to keep network interfaces in an expected order" > \ /etc/udev/rules.d/70-persistent-net.rules echo "" >> \ /etc/udev/rules.d/70-persistent-net.rules
cd /sys/class/net/ for NETDEV in $(ls | grep eth | sort) do ## Create a UDEV rule for each eth interface echo "## ${NETDEV} interface" >> \ /etc/udev/rules.d/70-persistent-net.rules
## We throw this one in here as it can contain some usefulinformation echo "## $(dmesg | grep ${NETDEV} | grep -i -v -e "console" -e "Command line" | head -1)" >> \ /etc/udev/rules.d/70-persistent-net.rules
echo -n "SUBSYSTEM==\"net\", ACTION==\"add\", DRIVERS==\"?*\","
\
/etc/udev/rules.d/70-persistent-net.rulesecho -n "ATTR{address}==\"$(cat ${NETDEV}/address)\", " >> \ /etc/udev/rules.d/70-persistent-net.rules echo -n "ATTR{dev_id}==\"0x0\", ATTR{type}==\"1\",KERNEL=="eth?*", " >> \ /etc/udev/rules.d/70-persistent-net.rules echo -e "NAME="${NETDEV}"\n" >> \ /etc/udev/rules.d/70-persistent-net.rules
## Make a log of the devices present during install echo -e "${NETDEV} $(cat ${NETDEV}/address)\n" >>/root/ksnet-devices
## Also remove the HWADDR line from all of the net config files grep -v -e NAME -e HWADDR -e NM_CONTROLLED \ /etc/sysconfig/network-scripts/ifcfg-${NETDEV} | sed's/"//g' \ > /etc/sysconfig/network-scripts/ifcfg-${NETDEV}-tmp echo "NM_CONTROLLED=no" >> /etc/sysconfig/network- scripts/ifcfg-${NETDEV}-tmp /usr/bin/perl -p -i -e 's/dhcp/none/' /etc/sysconfig/network- scripts/ifcfg-${NETDEV}-tmp mv -f /etc/sysconfig/network-scripts/ifcfg-${NETDEV}-tmp \ /etc/sysconfig/network-scripts/ifcfg-${NETDEV} done
###########################
On Wed, 25 Feb 2015 14:53:40 -0600, Ashley M. Kirchner ashley@pcraft.com wrote:
Thanks for that Jason but it didn't solve the problem. The system is still
coming up with the interfaces shuffled. It seems to *always* want to use the added ethernet card as eth0.
On Wed, Feb 25, 2015 at 1:37 PM, Jason Warr jason@warr.net wrote:
Starting back in RHEL/Cent 5 I found that the only way to make sure your
interface enumeration was consistent after install with what you had during install was to create a udev rules file using the mac addresses as the key. It is easy to run a short script in postinstall to create it based on how anaconda has seen them.
In order for this to work on Cent 6 you have to set biosdevname=0 on the kernel boot for the installed system.
PXE boot options:
label c6inst-sda kernel /linux-boot/cent6-x64/vmlinuz append initrd=/linux-boot/cent6-x64/initrd.img ksdevice=bootif ip=dhcp ks=http://xx.xx.xx.xx/install/linux/ks/basic-cent6-sda.cfg ipappend 2
In kickstart:
BOOTOPTS="biosdevname=0"
Also in kickstart I do not specify the config for ANY network interfaces. I let anaconda pull in only the config for the boot interface from PXE. I manually configure everything else. The only thing I do to non-boot interfaces is set the DHCP and ONBOOT to no.
On Wed, 25 Feb 2015 14:21:18 -0600, Ashley M. Kirchner < ashley@pcraft.com> wrote:
Version 6.6 ...
> > On Wed, Feb 25, 2015 at 1:17 PM, Jim Perrin jperrin@centos.org > wrote: > > <overly trimmed> > >> >> On 02/25/2015 01:56 PM, Ashley M. Kirchner wrote: >>> Ok, so some of this now works, but I'm still having problems. >>> With >> the >>> bootif option, the system now correctly configures and uses the >>> same >>> interface to get its kickstart file. However, when the system is >>> done >> and >>> boots up, the interfaces are still messed up. So this is what I >>> have >> in >> the >>> kickstart file: >> >> What version of CentOS 6 is this? >> >>> In the PXE config file I have: >>> >>> IPAPPEND 2 >>> APPEND ks=http://192.168.x.x/ks/portico.ks >> initrd=centos/x86_64/initrd.img >>> ramdisk_size=100000 ksdevice=bootif >> >>> As soon as I *remove* the additional ethernet card, the system >>> will >> boot >> up >>> with the ports configured correctly (port 1 = eth0, port 2 = >>> eth1). >> So >> why >>> is it that as soon as there is an additional one, all things go >>> to >> hell? >>> Why must the boot process shuffle them? More importantly, how do >>> I >> prevent >>> this so that the system comes up properly after a kickstart >>> install? >>> >> >> The reason I ask the version, is this is exactly the sort of >> thing that >> biosdevname is designed to solve. With biosdevname, you get >> devices >> like >> 'em1, em2, p6p1', which aren't as friendly as 'eth0' but also keep >> names >> sane and avoid the hair-tearing issues you're experiencing >> currently. >> You don't appear to be adding anything via your append line that >> would >> disable biosdevname, so I must assume you're using a much older 6 >> base >> install. >> >> >> In my experience biosdevname creates just as many problems as it solves. Dell can keep it.
-- >> Jim Perrin >> The CentOS Project | http://www.centos.org >> twitter: @BitIntegrity | GPG Key: FA09AD77 >> _______________________________________________ >> CentOS mailing list >> CentOS@centos.org >> http://lists.centos.org/mailman/listinfo/centos >> >> >> _______________________________________________ >> > CentOS mailing list > CentOS@centos.org > http://lists.centos.org/mailman/listinfo/centos > >
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Add "rdblacklist=MODULE_NAME" to your append line in pxelinux.conf file.
Trying that next. It'll have to wait till tomorrow as we're under a serious blizzard/snow event right now and I'd like to get home before all of hell freezes over. However, question, if I blacklist the module, that means during kickstart it doesn't know it's there either, which would cause a 'network' definition for that interface to fail (and kickstart to stop) ... I think.
This should get you into a usable state post-install so you can setup
additional interfaces.
My hope is that I don't have to do anything after it's done and reboots. The whole purpose of me doing this is in case something happens with the drive and I'm not here, someone can shove a new one in, reboot and let it go through the kickstart and be operational ten minutes later without them having done anything else. Maybe I'm aiming too damn high.
On Wed, 25 Feb 2015 17:11:02 -0600, Ashley M. Kirchner ashley@pcraft.com wrote:
Add "rdblacklist=MODULE_NAME" to your append line in pxelinux.conf file.
Trying that next. It'll have to wait till tomorrow as we're under a serious blizzard/snow event right now and I'd like to get home before all of hell freezes over. However, question, if I blacklist the module, that means during kickstart it doesn't know it's there either, which would cause a 'network' definition for that interface to fail (and kickstart to stop) ... I think.
It will if you try to configure the now non-existent interface.
This should get you into a usable state post-install so you can setup
additional interfaces.
My hope is that I don't have to do anything after it's done and reboots. The whole purpose of me doing this is in case something happens with the drive and I'm not here, someone can shove a new one in, reboot and let it go through the kickstart and be operational ten minutes later without them having done anything else. Maybe I'm aiming too damn high.
No, your not. One thing you might be doing though is being to concerned about ordering. The actual numbering of interfaces is, in most cases, purely cosmetic. So if you do not have another reason for needing a nice, logical ordering you should ignore it. Your use case is almost exactly why I went the route I did in the first place.
Since this appears to be a recovery plan for a specific system then your best course is to accept the ordering as kickstart sees it and adjust your network config lines to compensate. As long as you have a udev rules file the ordering will not change on reboot.
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On Feb 25, 2015 4:19 PM, "Jason Warr" jason@warr.net wrote:
It will if you try to configure the now non-existent interface.
That's what I figured, so I can remove it from the kickstart file, no problem. The question then becomes, if kickstart doesn't configure it, what happens when the system reboots after install? It won't know what to do with that interface, correct?
Is this a case where I will need to put an ifcfg-eth2 file in place during post-install?
On Wed, 25 Feb 2015 17:30:30 -0600, Ashley M. Kirchner ashley@pcraft.com wrote:
On Feb 25, 2015 4:19 PM, "Jason Warr" jason@warr.net wrote:
It will if you try to configure the now non-existent interface.
That's what I figured, so I can remove it from the kickstart file, no problem. The question then becomes, if kickstart doesn't configure it, what happens >when the system reboots after install? It won't know what to do with that interface, correct?
Is this a case where I will need to put an ifcfg-eth2 file in place during post-install?
Upon reboot the system *should* generate a base one for you as it will see it as a new interface. Not a big deal if it does not though, just create one yourself. You will want to add it to the udev rules file though. You can re-run the script I sent to do that if you want. At that point it should be eth2. Or you can edit the existing one by copying a line and changing the MAC and eth* to whatever you need.
On Wed, Feb 25, 2015 at 4:45 PM, Ashley M. Kirchner ashley@pcraft.com wrote:
Out of order meaning, it puts the additional ethernet card as eth0, with the built-in ports as eth1 and eth2 respectively. WITHOUT the additional card installed, it puts the built-in ports as eth0 and eth1, which is what I want it to do. The additional card should be eth2, at least that's what I want it to do.
Looking at persistent-net.rules, it always puts the additional card first, both without your script as well as with. I need it to be last. The system's built-ins should always be eth0 and eth1 respectively.
How can you confirm 'always' here? I haven't done it in the context of PXE booting, but I see random ordering of cards and motherboard nics on first installs with Centos6.x That is, the nics on the motherboard and on each card will have adjacent numbers but the cards and motherboard sets jump around until the MAC addresses are recorded in the etc/udev/rules.d/70-persistent-net.rules to nail them down.
I have a Dell server that has two built-in ethernet devices. When I kickstart the machine, they are correctly identified as eth0 and eth1 (correctly meaning they correspond to the physical device ports 1 and 2). I need a third one and want that to come up as eth2. After adding the hardware, kickstart now fails because for some reason it goes through a rename process where it makes the newly added card eth1 (or eth0, I forgot). Is there a way to stop this rename process so kickstart correctly uses the physical hardware the way they are, meaning physical port 1 = eth0, port 2 = eth1, and the additional ethernet card then becomes eth2?
Should I be using the device's MAC address when I set the 'network' option in the kickstart file? So instead of 'network --device=eth0' I make it 'network -device=aa;bb:cc:dd:eee:ff' ?
ksdevice=aa;bb:cc:dd:ee:ff in your above example will ensure the device with that mac is the kickstart device.
On Thu, Feb 26, 2015 at 3:42 PM, Tom Brown tom@eazyriders.no-ip.biz wrote:
ksdevice=aa;bb:cc:dd:ee:ff in your above example will ensure the device with that mac is the kickstart device.
Yeah, turned out bootif accomplishes the same thing, at least in my scenario. What happened afterwards though is different.
-
Ashley M. Kirchner -
Digimer -
Ian Pilcher -
Jason Warr -
Jim Perrin -
Les Mikesell -
m.roth＠5-cent.us -
Steven Tardy -
Tom Brown