I have a Centos 5 machine here that, up until about a year ago, was happily running Icecast and serving streaming audio through through three network connections, consisting of one "local" connection (local address 192.168.1.5) and two cable modems to talk to the outside world.
We shut this down about a year ago, but now I am attempting to get it going again on one outside connection instead of two. Simply changing the IP address on one of the network cards to use the new address isn't working, so the notes that I made when I set this thing up in the first place must be incomplete since I'm obviously missing something here.
I have two "active" IP addresses that I want to use at the moment.
eth0 is 192.168.1.5 and is working fine. I can log into the server with ssh and run icecast and listen to streaming audio just fine.
eth1 is now supposed to be 204.83.105.1 so I configured the new address on that card with system-config-network.
The third card (eth2) I plan to ignore for the time being so I haven't changed that or done anything with it at all.
My /etc/iproute2/rt_tables looks like this. I haven't changed it from what it was when I originally set this thing up a few years ago.
# # reserved values # 255 local 254 main 253 default 0 unspec # # local # #1 inr.ruhep 50 access1 60 access2
Note the access1 and access2 entries at the bottom of the file.
I then ran the following three commands:
ip route add 204.83.15.0/24 dev eth1 table access1 ip route add default via 204.83.15.254 dev eth1 table access1 ip rule add from 204.83.15.1/32 lookup access1
These are the same commands that I used to set up the previous Internet connection on this server -- The only thing that I have changed was the IP addresses.
Here is the output from "ip route show":
[root@audio ~]# ip route show 192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.5 204.83.15.0/24 dev eth1 proto kernel scope link src 204.83.15.1 169.254.0.0/16 dev eth1 scope link default via 204.83.15.254 dev eth1
That seems to indicate that the default route is 204.83.15.254 which is the correct gateway number for that Internet connection.
However, a ping or traceroute command (ping google.com or whatever) gives me no output.
I've missed a step somewhere.
I hooked my laptop up to that Internet connection to insure that the modem and whatnot is working and it is, so there's obviously something wrong with my configuration.
Can any of you folks tell me what I've missed?
On Sat, 18 Jul 2015 11:41:53 -0600 Frank Cox wrote: output.
I've missed a step somewhere.
I hooked my laptop up to that Internet connection to insure that the modem and whatnot is working and it is, so there's obviously something wrong with my configuration.
Can any of you folks tell me what I've missed?
Here's something interesting.
When I run a traceroute to 204.83.15.254 I get this:
[frankcox@audio ~]$ traceroute 204.83.15.254 traceroute to 204.83.15.254 (204.83.15.254), 30 hops max, 40 byte packets 1 (204.83.15.1) 3000.077 ms !H 3000.068 ms !H 3000.052 ms !H
It thinks that 204.83.15.254 is down, and that's the gateway address for eth1 that I want to be the default gateway.
And 192.168.1.1 is the address of the router that eth0 is plugged into:
[frankcox@audio network-scripts]$ ping 192.168.1.1 PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data. --- 192.168.1.1 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 1999ms
I can ssh into the server through that router (and eth0) with no problem. That's how I'm communicating with it right now.
[frankcox@audio network-scripts]$ /sbin/route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 204.83.15.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1 169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1 0.0.0.0 204.83.15.254 0.0.0.0 UG 0 0 0 eth1
On 07/18/2015 10:41 AM, Frank Cox wrote:
I then ran the following three commands:
ip route add 204.83.15.0/24 dev eth1 table access1 ip route add default via 204.83.15.254 dev eth1 table access1 ip rule add from 204.83.15.1/32 lookup access1
...
Can any of you folks tell me what I've missed?
Does the system work correctly if you don't run those "ip route" commands?
On Sat, 18 Jul 2015 21:34:27 -0700 Gordon Messmer wrote:
Does the system work correctly if you don't run those "ip route" commands?
It's exactly the same. And what I mean by exactly the same is EXACTLY the same.
I rebooted the system which should clear out my route commands and whatnot, and discovered that everything (route -n and ip route show and whatnot) are exactly the same as before. Therefore, the ip route and ip rule commands apparently had no effect whatsoever -- what I got after entering the ip route commands is exactly what I have without entering those commands.
Interesting. But since it's still exactly the same, it's still not working; it still fails in exactly the same way too.
On 07/18/2015 10:12 PM, Frank Cox wrote:
Interesting. But since it's still exactly the same, it's still not working; it still fails in exactly the same way too.
Yes, but that means you need to start with the standard troubleshooting stuff. Do you have link? Is the Ethernet cable working? Do you see any traffic if you run "tcpdump -nn -i eth1"? Double-check your IP address and gateway in the configuration file. Is the gateway's MAC address listed in the output of "arp"?
On Sat, 18 Jul 2015 22:37:30 -0700 Gordon Messmer wrote:
On 07/18/2015 10:12 PM, Frank Cox wrote:
Interesting. But since it's still exactly the same, it's still not working; it still fails in exactly the same way too.
Yes, but that means you need to start with the standard troubleshooting stuff. Do you have link? Is the Ethernet cable working?
[root@audio ~]# ip link 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000 link/ether 00:26:5a:07:f0:da brd ff:ff:ff:ff:ff:ff 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000 link/ether 00:26:5a:07:ef:8d brd ff:ff:ff:ff:ff:ff 4: eth2: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast qlen 1000 link/ether 00:40:05:86:66:4a brd ff:ff:ff:ff:ff:ff 5: sit0: <NOARP> mtu 1480 qdisc noop link/sit 0.0.0.0 brd 0.0.0.0
I double-checked that I had the ethernet cable plugged into the right port on the server by unplugging it and then eth1 said "NO CARRIER". As far as I know, that means that the cable and port are working.
Do you see > any traffic if you run "tcpdump -nn -i eth1"?
I see no traffic on eth1 with that command until I log into another session and type "ping google.com". Then I get this output:
[root@audio ~]# tcpdump -nn -i eth1 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 96 bytes 00:11:00.412188 arp who-has 204.83.15.254 tell 204.83.15.1 00:11:01.412135 arp who-has 204.83.15.254 tell 204.83.15.1 00:11:02.412112 arp who-has 204.83.15.254 tell 204.83.15.1
3 packets captured 3 packets received by filter 0 packets dropped by kernel
Double-check your IP address and gateway in the configuration file.
Seems to be correct.
Is the gateway's MAC address listed in the output of "arp"?
Apparently not.
[root@audio ~]# arp Address HWtype HWaddress Flags Mask Iface 192.168.1.1 ether 00:24:01:F3:93:21 C eth0 204.83.15.254 (incomplete) eth1
I don't know what that means; this is the first time I ever typed the arp command.
Am 19.07.2015 um 08:13 schrieb Frank Cox:
On Sat, 18 Jul 2015 22:37:30 -0700 Gordon Messmer wrote:
[ ... ]
Do you see > any traffic if you run "tcpdump -nn -i eth1"?
I see no traffic on eth1 with that command until I log into another session and type "ping google.com". Then I get this output:
[root@audio ~]# tcpdump -nn -i eth1 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 96 bytes 00:11:00.412188 arp who-has 204.83.15.254 tell 204.83.15.1 00:11:01.412135 arp who-has 204.83.15.254 tell 204.83.15.1 00:11:02.412112 arp who-has 204.83.15.254 tell 204.83.15.1
No response to the arp queries.
3 packets captured 3 packets received by filter 0 packets dropped by kernel
Double-check your IP address and gateway in the configuration file.
Seems to be correct.
Is the gateway's MAC address listed in the output of "arp"?
Apparently not.
[root@audio ~]# arp Address HWtype HWaddress Flags Mask Iface 192.168.1.1 ether 00:24:01:F3:93:21 C eth0 204.83.15.254 (incomplete) eth1
Again, no ARP result.
I don't know what that means; this is the first time I ever typed the arp command.
Clearly your gateway 204.83.15.254 does not act like it should. Look broken or misconfigured, at least from within your network.
Alexander
On Sun, 19 Jul 2015 17:27:09 +0200 Alexander Dalloz wrote:
Clearly your gateway 204.83.15.254 does not act like it should. Look broken or misconfigured, at least from within your network.
This server has three network cards in it. I just disabled (unconfigured) eth1 and configured eth2 with the external IP address and whatnot. Plugged in the modem to eth2 and away she goes. Everything works as it should. Cranked up the streaming audio and rock on!
So I apparently have a hardware issue; eth1 has failed in some strange way. Oh well. I'm not going to worry about replacing it right away since I'm just using two network connections at the moment anyway.
I'm running a yum update on that machine right now (for the first time since it was shut off a year ago) and all appears to be well.
Thanks to everyone for taking the time to look at my issue and provide input.
I hate mysterious hardware failures.
The only bad news is that the radio station has fallen off the air today for some reason, so all I have to stream from there is static. Oh well -- that end is NOT my problem.
Thanks again for the assistance, folks!
On Sun, Jul 19, 2015 at 01:31:16PM -0600, Frank Cox wrote:
On Sun, 19 Jul 2015 17:27:09 +0200 Alexander Dalloz wrote:
Clearly your gateway 204.83.15.254 does not act like it should. Look broken or misconfigured, at least from within your network.
This server has three network cards in it. I just disabled (unconfigured) eth1 and configured eth2 with the external IP address and whatnot. Plugged in the modem to eth2 and away she goes. Everything works as it should. Cranked up the streaming audio and rock on!
So I apparently have a hardware issue; eth1 has failed in some strange way. Oh well. I'm not going to worry about replacing it right away since I'm just using two network connections at the moment anyway.
Have you tried shutting down all the way to power-off and doing a full cold reboot? I've experienced (rare) cases where some bit of HW getes wedged and won't reset except for a cold boot.
On Sun, 19 Jul 2015 20:18:07 -0400 Fred Smith wrote:
Have you tried shutting down all the way to power-off and doing a full cold reboot? I've experienced (rare) cases where some bit of HW getes wedged and won't reset except for a cold boot.
No, I haven't done that. I rebooted it several times but not from cold boot.
It's an idea, though.
On 07/18/2015 11:13 PM, Frank Cox wrote:
[root@audio ~]# tcpdump -nn -i eth1 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 96 bytes 00:11:00.412188 arp who-has 204.83.15.254 tell 204.83.15.1 00:11:01.412135 arp who-has 204.83.15.254 tell 204.83.15.1
...
Is the gateway's MAC address listed in the output of "arp"?
[root@audio ~]# arp Address HWtype HWaddress Flags Mask Iface 204.83.15.254 (incomplete) eth1
I don't know what that means; this is the first time I ever typed the arp command.
When you're using Ethernet, packets are transmitted between cards using the MAC address of the recipient's interface. IPv4 resolves hardware addresses (MAC address) using the ARP protocol. In order to send a packet to 204.83.15.254, a host on the same network segment sends a broadcast request (arp who-has) request for the hardware address associated with that IPv4 address. The host with that IPv4 address should send a unicast reply to the host that sent the request. Understanding arp is essential to troubleshooting IPv4 and Ethernet. (IPv6 does not use ARP to resolve MAC addresses)
Your host, 204.83.15.1 is on a /24 network with its gateway, 204.83.15.254. I would expect that a /24 network would probably have more than two hosts. If that's the case, it would be extremely unusual to see no broadcast traffic when you run "tcpdump" on that interface. Normally you'd see arp broadcast requests every few seconds, even if you didn't see any other traffic.
It's hard to say specifically what the problem might be without knowing more about the physical topology of your network, but the most likely problems are that you're connected to a network segment with no other hosts, or that you're on a segment with only one host (the gateway) which has no need to broadcast anything and is on a different address than you expect, or that your cable is defective (even with link), or that the device your host is physically attached to is defective.
-
Alexander Dalloz -
Frank Cox -
Fred Smith -
Gordon Messmer