Hello all
Here is the scenario: We have a mail server VM which currently has two virtual NICs attached to it. One NIC is has an IP on a subnet with a default gateway defined and the other NIC has an IP on a different subnet with a different gateway on a different VLAN defined. Now when I activate both NICs, and run an ifconfig -a, I see that both IP addresses are showing. Now here is the problem. When I ping the VM, the first NIC's IP is not pingable at all, but the second NIC's IP is pingable. How do I configure the networking so that both IPs are pingable and the VM is reachable via both IPs? Please let me know what I may be doing wrong!
Thank you!
Sean
On 07/11/16 02:00 AM, Sean Son wrote:
Hello all
Here is the scenario: We have a mail server VM which currently has two virtual NICs attached to it. One NIC is has an IP on a subnet with a default gateway defined and the other NIC has an IP on a different subnet with a different gateway on a different VLAN defined. Now when I activate both NICs, and run an ifconfig -a, I see that both IP addresses are showing. Now here is the problem. When I ping the VM, the first NIC's IP is not pingable at all, but the second NIC's IP is pingable. How do I configure the networking so that both IPs are pingable and the VM is reachable via both IPs? Please let me know what I may be doing wrong!
We're missing a lot of details, but to speak generally, I'd look at the bridging on the host if you're confident that the interfaces are configured properly inside the host.
Also, have you looked at iptables?
On Mon, 7 Nov 2016 02:00:24 -0500 Sean Son wrote:
When I ping the VM, the first NIC's IP is not pingable at all, but the second NIC's IP is pingable. How do I configure the networking so that both IPs are pingable and the VM is reachable via both IPs?
The last time I did that I did it on Centos 5. Here are my notes from back then.
First, add two lines to /etc/iproute2/rt_tables, so it looks like this:
# # reserved values # 255 local 254 main 253 default 0 unspec # # local # #1 inr.ruhep 50 access1 60 access2
Then add the following routes:
ip route add 1.2.3.0/24 dev eth1 table access1 ip route add default via 1.2.3.1 dev eth1 table access1 ip rule add from 1.2.3.178/32 lookup access1
ip route add 1.2.3.0/24 dev eth2 table access2 ip route add default via 1.2.3.1 dev eth2 table access2 ip rule add from 1.2.3.180/32 lookup access2
Of course, in this example 1.2.3.178 is the IP address of the first nic and 1.2.3.180 is the address of the second nic.
On 7/11/2016 6:00 PM, Sean Son wrote:
Hello all
Here is the scenario: We have a mail server VM which currently has two virtual NICs attached to it. One NIC is has an IP on a subnet with a default gateway defined and the other NIC has an IP on a different subnet with a different gateway on a different VLAN defined. Now when I activate both NICs, and run an ifconfig -a, I see that both IP addresses are showing. Now here is the problem. When I ping the VM, the first NIC's IP is not pingable at all, but the second NIC's IP is pingable. How do I configure the networking so that both IPs are pingable and the VM is reachable via both IPs? Please let me know what I may be doing wrong!
I've done this recently on C7. The key words for your Google Fu is policy based routing. That ensures the incoming packets come back out the same interface they came in on. Mind you, depending on the routers in the way this may not be a problem. CISCO ASAs get really picky but otherwise there may be other problems to deal with as well.
If you're using Network Manager then the package: NetworkManager-config-routing-rules will help.
After that you create the rules and routes files in /etc/sysconfig/network-scripts
So in my case I have:
* enp3s0: 192.168.20.253 (the default gateway on this subnet is 192.168.20.2) * vlan22@enp3s0: 192.168.22.253 (the default gateway on this subnet is 192.168.22.1)
[root@host network-scripts]# cat route-enp3s0 192.168.20.0/24 dev enp3s0 tab 20 default via 192.168.20.2 dev enp3s0 tab 20
[root@host network-scripts]# cat rule-enp3s0 from 192.168.20.0/24 tab 20 priority 20
[root@host network-scripts]# cat route-vlan22 192.168.22.0/24 dev vlan22 tab 22 default via 192.168.22.1 dev vlan22 tab 22
[root@host network-scripts]# cat rule-vlan22 from 192.168.22.0/24 tab 22 priority 22
Wireshark can be really helpful here as it can confirm that the "unpingable" NIC is actually receiving the packets that you think it is. And hopefully in that case can also confirm if the echo packets are going out the alternate NIC. If there are no incoming packets then the PBR is no real help at all and you'd need to look further upstream.
Hope that helps, -pete
On 11/06/2016 11:00 PM, Sean Son wrote:
How do I configure the networking so that both IPs are pingable and the VM is reachable via both IPs?
You need one rule file per interface, which directs traffic out the appropriate interface based on the source address of the packet:
https://blogs.oracle.com/networking/entry/advance_routing_for_multi_homed
Hello there,
What is the hypervisor that hosts the VM? What does ifconfig show on it?
Boris.
On Mon, Nov 7, 2016 at 4:36 PM, Gordon Messmer gordon.messmer@gmail.com wrote:
On 11/06/2016 11:00 PM, Sean Son wrote:
How do I configure the networking so that both IPs are pingable and the VM is reachable via both IPs?
You need one rule file per interface, which directs traffic out the appropriate interface based on the source address of the packet:
https://blogs.oracle.com/networking/entry/advance_routing_for_multi_homed
CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
On Tue, Nov 8, 2016 at 8:02 AM, Boris Epstein borepstein@gmail.com wrote:
Hello there,
What is the hypervisor that hosts the VM? What does ifconfig show on it?
Boris.
On Mon, Nov 7, 2016 at 4:36 PM, Gordon Messmer gordon.messmer@gmail.com wrote:
On 11/06/2016 11:00 PM, Sean Son wrote:
How do I configure the networking so that both IPs are pingable and the VM is reachable via both IPs?
You need one rule file per interface, which directs traffic out the appropriate interface based on the source address of the packet:
routing_for_multi_homed
CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Hello all
Thank you for the replies. I ran into some issues with the policy based routing which I will explain in a few. First off to answer each one of your questions:
Digimer: No I didnt take a look at IPTables
Peter Brady: Thank you for the example, I tried that but it failed and I will explain in a minute.
Frank Cox : That works but how do I make it persistent across reboots?
Boris Epstein: I am using Hyper-V and its getting annoying lol
Ok so here is how I have set everything up:
my /etc/iproute2/rt_tables:
# # reserved values # 255 local 254 main 253 default 0 unspec # # local # #1 inr.ruhep 300 NIC1 310 NIC2
my /etc/sysconfig/network-scripts/route-eth0 file:
168.87.147.0/24 dev eth0 table 300 default via 168.87.147.1 dev eth0 table 300
/etc/sysconfig/network-scripts/route-eth1 file:
10.20.50.0/24 dev eth1 table 310 default via 10.20.50.1 dev eth1 table 310
My /etc/sysconfig/network-scripts/rule-eth0:
from 168.87.147.33/32 table 300 to 168.87.147.33 table 300
My /etc/sysconfig/network-scripts/rule-eth1:
from 10.20.50.90/32 table 310 to 10.20.50.90 table 310
now after implementing this and restarting NetworkManager, when I run 'ip rule list', I get the following:
0: from all lookup local 32764: from all to 10.20.50.90 lookup NIC2 32765: from 10.20.50.90 lookup NIC2 32766: from all lookup main 32767: from all lookup default
and when i run 'ip route' , i get the following:
default via 168.87.147.1 dev eth0 proto static metric 1024 10.20.50.0/24 dev eth1 proto kernel scope link src 10.20.50.90 168.87.147.0/24 dev eth0 proto kernel scope link src 168.87.147.33
yet when I try to ping from another server to 10.20.50.90 it will not ping at all. Also, whenever I reboot the VM, eth1 switches over to DHCP and I lose my IP configuration. After I reset the IP Configuration back to Manual and reenter the IP, Centos creates a new interface file called ifcfg-Wired_Connection-1 and places the IP configuration for the interface into that file. Both virtual NICs are set to Static Mac Addresses, so I dont know why it keeps creating another interface file. Any ideas on what I am doing wrong here? All help is greatly appreciated!
Thanks!
On 11/14/2016 12:47 PM, Sean Son wrote:
Any ideas on what I am doing wrong here?
Nothing obvious. Since your interfaces have static configurations, I'd suggest turning off NetworkManager and turning on the "network" service to determine whether or not that works correctly after a reboot.
Assuming it does, fixing the issue might be as simple as getting NetworkManager to not rename one interface to "wired connection 1".
On Mon, Nov 14, 2016 at 5:30 PM, Gordon Messmer gordon.messmer@gmail.com wrote:
On 11/14/2016 12:47 PM, Sean Son wrote:
Any ideas on what I am doing wrong here?
Nothing obvious. Since your interfaces have static configurations, I'd suggest turning off NetworkManager and turning on the "network" service to determine whether or not that works correctly after a reboot.
Assuming it does, fixing the issue might be as simple as getting NetworkManager to not rename one interface to "wired connection 1".
CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Hello Gordon
Thank you for your response. I disabled NetworkManager and started the network service, then I rebooted the VM and now 'ip rule show' has the rules for both interfaces showing but I have no network connectivity even when I restart the network service. Should I reenable NetworkManager now? How would I go about getting NetworkManager to stop creating new Interface files and naming them Wired_connection-*?
Thanks!
On 11/15/2016 06:07 AM, Sean Son wrote:
I have no network connectivity even when I restart the network service. Should I reenable NetworkManager now?
Yeah, the switch is just a test to see if the problem is specific to NetworkManager. It seems that you have other problems as well.
Before you do that, post the output of the following:
ip route show
ip route show table 300
ip route show table 301
ip rule show
cat /etc/sysconfig/network-scripts/ifcfg-eth0
cat /etc/sysconfig/network-scripts/route-eth0
cat /etc/sysconfig/network-scripts/rule-eth0
cat /etc/sysconfig/network-scripts/ifcfg-eth1
cat /etc/sysconfig/network-scripts/route-eth1
cat /etc/sysconfig/network-scripts/rule-eth1
On Tue, Nov 15, 2016 at 10:55 AM, Gordon Messmer gordon.messmer@gmail.com wrote:
On 11/15/2016 06:07 AM, Sean Son wrote:
I have no network connectivity even when I restart the network service. Should I reenable NetworkManager now?
Yeah, the switch is just a test to see if the problem is specific to NetworkManager. It seems that you have other problems as well.
Before you do that, post the output of the following:
ip route show
ip route show table 300
ip route show table 301
ip rule show
cat /etc/sysconfig/network-scripts/ifcfg-eth0
cat /etc/sysconfig/network-scripts/route-eth0
cat /etc/sysconfig/network-scripts/rule-eth0
cat /etc/sysconfig/network-scripts/ifcfg-eth1
cat /etc/sysconfig/network-scripts/route-eth1
cat /etc/sysconfig/network-scripts/rule-eth1
CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Hello
I think i figured it out. I had forgotten to install NetworkManager-config-routing-rules and then enable and start the NetworkManager-dispatcher service. I tried that on a different test machine and I am able to ping both of the IPs from another machine just fine. Only a few things I have noticed: No matter how many times I restart the NetworkManager-dispatcher service, if I reboot the system, and run a systemctl status NetworkManager-dispatcher.service, the service shows as Inactive (Dead) even though it has been enabled. Also whenever I reboot the VM, after logging into the GUI, I get a pop up that says failed to activate network connection , yet I still have network connectivity. Any ideas on what I can do to fix these things? Also overall, because I am using a static IP config for both NICs, is it better to just disable NetworkManager and use network scripts instead?
Thanks!!
On 11/16/2016 07:33 AM, Sean Son wrote:
No matter how many times I restart the NetworkManager-dispatcher service, if I reboot the system, and run a systemctl status NetworkManager-dispatcher.service, the service shows as Inactive (Dead) even though it has been enabled.
That's normal. It's a dbus-invoked service. You don't need to manually enable it. It runs a command when it's invoked, and then terminates. There isn't a daemon process associated with the service.
Also whenever I reboot the VM, after logging into the GUI, I get a pop up that says failed to activate network connection , yet I still have network connectivity. Any ideas on what I can do to fix these things?
Not really. The log might tell you why it thinks an interface wasn't successfully enabled.
Also overall, because I am using a static IP config for both NICs, is it better to just disable NetworkManager and use network scripts instead?
Not generally. If you find the old network service works better for your use case, it's available. I use it on a small number of hosts where NM bugs interfere with normal operation, but I'm waiting for the 7.3 updates (where NM has been rebased to a newer version) to see if I can stop doing that.
On 07/11/16 07:00, Sean Son wrote:
Hello all
Here is the scenario: We have a mail server VM which currently has two virtual NICs attached to it. One NIC is has an IP on a subnet with a default gateway defined and the other NIC has an IP on a different subnet with a different gateway on a different VLAN defined. Now when I activate both NICs, and run an ifconfig -a, I see that both IP addresses are showing. Now here is the problem. When I ping the VM, the first NIC's IP is not pingable at all, but the second NIC's IP is pingable. How do I configure the networking so that both IPs are pingable and the VM is reachable via both IPs? Please let me know what I may be doing wrong!
Thank you!
Sean _______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
policy based routing is useful concept but in my mind should be the very last resort. It is the kernel's networking functions which are responsible for routing tables construction and if you can shed more details (like mentioned earlier) on what can(not) ping what(VM hosting box <-> VM guest / outside internal net <-> VM / etc.) centbuddies could share more concrete thoughts then. Basically - with policy/source based routing you need to be very careful, test everything - best is to leave it to the kernel and only use main table, kernel should be the smartest, for more complex setups maybe think "routing daemons".
-
Boris Epstein -
Digimer -
Frank Cox -
Gordon Messmer -
lejeczek -
Peter Brady -
Sean Son