The web site for CentOS is not responding. I can ping it OK, but the web site is not responding to http requests from my browser. If the webmaster should happen to read this...
On Mon, 16 Mar 2009, Lanny Marcus wrote:
The web site for CentOS is not responding. I can ping it OK, but the web site is not responding to http requests from my browser. If the webmaster should happen to read this...
working fine here, and from two other test locations -- it might make more sense to use the IRC channel for time sensitive reports
-- Russ herrold
On 3/16/09, R P Herrold herrold@centos.org wrote:
On Mon, 16 Mar 2009, Lanny Marcus wrote:
The web site for CentOS is not responding. I can ping it OK, but the web site is not responding to http requests from my browser. If the webmaster should happen to read this...
working fine here, and from two other test locations -- it might make more sense to use the IRC channel for time sensitive reports
OK. Good. It may be a temporary connectivity problem, to sites located in certain areas of the USA. Still not loading for me.
Always useful to check with: http://downforeveryoneorjustme.com/ http://internetpulse.net/ http://just-ping.com
Before reporting connectivity problems.
Patrick
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Lanny Marcus Sent: Monday, March 16, 2009 1:39 PM To: CentOS mailing list Subject: Re: [CentOS] OT: centos.org web site not responding
On 3/16/09, R P Herrold herrold@centos.org wrote:
On Mon, 16 Mar 2009, Lanny Marcus wrote:
The web site for CentOS is not responding. I can ping it
OK, but the
web site is not responding to http requests from my
browser. If the
webmaster should happen to read this...
working fine here, and from two other test locations -- it
might make
more sense to use the IRC channel for time sensitive reports
OK. Good. It may be a temporary connectivity problem, to sites located in certain areas of the USA. Still not loading for me. _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On 3/16/09, Flaherty, Patrick pflaherty@wsi.com wrote:
Always useful to check with: http://downforeveryoneorjustme.com/ http://internetpulse.net/ http://just-ping.com
Before reporting connectivity problems.
Cool Pat. I'd never heard of any of those sites before. Thanks. I clicked on your first link, tested CentOS and it's "just me". Not sure what the problem is, but hopefully it will go away.
R P Herrold wrote:
On Mon, 16 Mar 2009, Lanny Marcus wrote:
The web site for CentOS is not responding. I can ping it OK, but the web site is not responding to http requests from my browser. If the webmaster should happen to read this...
working fine here, and from two other test locations -- it might make more sense to use the IRC channel for time sensitive reports
yeah, from here and from my work, too, which are on two very different networks.
On 3/16/09, John R Pierce pierce@hogranch.com wrote:
R P Herrold wrote:
On Mon, 16 Mar 2009, Lanny Marcus wrote:
The web site for CentOS is not responding. I can ping it OK, but the web site is not responding to http requests from my browser. If the webmaster should happen to read this...
working fine here, and from two other test locations -- it might make more sense to use the IRC channel for time sensitive reports
yeah, from here and from my work, too, which are on two very different networks.
I'm glad it's working for you and Russ. Must be a routing problem from here (I use opendns.com). I can get to my web sites, which are on servers in CT, without any problems, so it seems to be a geographic thing, from my end. Whatever the problem is, the good news is that the CentOS web site is up and running OK for you. :-)
On Mon, Mar 16, 2009 at 12:29 PM, R P Herrold herrold@centos.org wrote:
On Mon, 16 Mar 2009, Lanny Marcus wrote:
The web site for CentOS is not responding. I can ping it OK, but the web site is not responding to http requests from my browser. If the webmaster should happen to read this...
working fine here, and from two other test locations -- it might make more sense to use the IRC channel for time sensitive reports
Russ: After Layered Tech eliminated the problem, I saved the web page about CentOS on IRC to my hard drive. If I ever run into something like that again, I'm prepared now.... Lanny
On Mon, Mar 16, 2009 at 12:29 PM, R P Herrold herrold@centos.org wrote:
On Mon, 16 Mar 2009, Lanny Marcus wrote:
The web site for CentOS is not responding. I can ping it OK, but the web site is not responding to http requests from my browser. If the webmaster should happen to read this...
working fine here, and from two other test locations -- it might make more sense to use the IRC channel for time sensitive reports
Russ: Same problem with the centos.org web site again from my end. I did go to IRC, as you suggested, and sent a message to #centos-web several minutes ago. Didn't see anyone else there. I hope someone will reopen the Trouble Ticket at Layered Tech. Lanny
On Thu, Mar 19, 2009 at 02:07:15PM -0500, Lanny Marcus wrote:
Russ: Same problem with the centos.org web site again from my end. I did go to IRC, as you suggested, and sent a message to #centos-web several minutes ago. Didn't see anyone else there. I hope someone will reopen the Trouble Ticket at Layered Tech. Lanny
<quoting LT support on re-opened ticket> From the traceroute you provided, the only sign of a potential problem appears to be on Level3's network in Dallas. I should also clarify that we did not fix the issue your user was experiencing a few days ago, as we were unable to duplicate the problem.
I just ran a test from just-ping.com to your web server, and it looks very good from over 30 locations worldwide.
Thank you,
Network Operations Layered Technologies </quote>
Tru
2009/3/19 Tru Huynh tru@centos.org:
On Thu, Mar 19, 2009 at 02:07:15PM -0500, Lanny Marcus wrote:
Russ: Same problem with the centos.org web site again from my end. I did go to IRC, as you suggested, and sent a message to #centos-web several minutes ago. Didn't see anyone else there. I hope someone will reopen the Trouble Ticket at Layered Tech. Lanny
<quoting LT support on re-opened ticket> From the traceroute you provided, the only sign of a potential problem appears to be on Level3's network in Dallas. I should also clarify that we did not fix the issue your user was experiencing a few days ago, as we were unable to duplicate the problem.
Tru: Thank you for reopening the trouble ticket at Layered Tech and posting their reply! John Pierce found there was a problem, in Dallas, when he did a lot of ping tests to centos.org the night of the 16th. There is a problem and if it isn't within Layered Tech, it must be at Level3 or where they connect. The other day, everyone else was able to load the web site, when I couldn't, so there must be something about our connection that makes the problem visible to me, but not to others. Or, that it involves something on the Level3 route from Miami to Dallas. However, John Pierce was also going in on Level3, from San Jose to Dallas, and he discovered a problem. I would assume, if I assume, that Layered Tech is the one who can report the problem to Level3. Possibly the problem is where Level3 does the interconnection to Layered Tech? This is what John Pierce posted the night of the 16th:
"I did some more pinging later on from one of my servers in San Jose, California, and saw some signs of route instability at layeredwhazza where the next hop after what should have been the next-to-last one was coming from various routers all going 'no route to destination' "
and John Stanley wrote on the 17th:
"But doing another trace to layeredtech and ltdomains shows bad hops also from my end just like John Pierces shows. But still I can access it by WWW."
I just ran a test from just-ping.com to your web server, and it looks very good from over 30 locations worldwide.
He's right about that. I just pinged centos.org but the web pages are not available to me, but are available to others on the list. Below is a traceroute I just did. Notice hops 8 and 9 are not shown. Lanny
[lanny@dell2400 ~]$ traceroute centos.org traceroute to centos.org (72.232.194.162), 30 hops max, 40 byte packets 1 ipcop233 (192.168.10.1) 0.516 ms 0.513 ms 0.513 ms 2 192.168.1.1 (192.168.1.1) 1.397 ms 1.816 ms 1.592 ms 3 dsl-emcali-190.1.248.1.emcali.net.co (190.1.248.1) 16.909 ms 19.273 ms 21.105 ms 4 172.16.1.3 (172.16.1.3) 23.123 ms 14.492 ms * 5 190.90.2.25 (190.90.2.25) 55.225 ms 60.168 ms 60.299 ms 6 so-4-2-1-nmi-core01.nwnnetwork.net (63.245.40.149) 97.314 ms 99.549 ms 109.844 ms 7 ge-1-1-0-nmi-core02.columbus-networks.com (63.245.5.0) 105.428 ms 106.990 ms 109.280 ms 8 * * * 9 * * * 10 ae-2.ebr1.Dallas1.Level3.net (4.69.140.133) 118.310 ms 115.144 ms 117.366 ms 11 ae-61-61.csw1.Dallas1.Level3.net (4.69.136.122) 119.042 ms ae-81-81.csw3.Dallas1.Level3.net (4.69.136.130) 120.057 ms ae-71-71.csw2.Dallas1.Level3.net (4.69.136.126) 129.584 ms 12 * ae-42-99.car2.Dallas1.Level3.net (4.68.19.196) 120.725 ms 123.532 ms 13 DATABANK-HO.car2.Dallas1.Level3.net (4.71.170.2) 115.159 ms 115.091 ms 117.425 ms 14 aw_cw_10g.databank.com (63.164.96.54) 118.347 ms 116.937 ms 117.710 ms 15 pod22c_ae.layeredtech.com (63.164.96.202) 117.607 ms 117.466 ms pod22a_aw.layeredtech.com (63.164.96.242) 115.099 ms 16 162.194.232.72.static.reverse.ltdomains.com (72.232.194.162) 118.377 ms !X 115.107 ms !X 117.395 ms !X [lanny@dell2400 ~]$
2009/3/19 Tru Huynh tru@centos.org:
On Thu, Mar 19, 2009 at 02:07:15PM -0500, Lanny Marcus wrote:
Russ: Same problem with the centos.org web site again from my end. I did go to IRC, as you suggested, and sent a message to #centos-web several minutes ago. Didn't see anyone else there. I hope someone will reopen the Trouble Ticket at Layered Tech. Lanny
<quoting LT support on re-opened ticket> From the traceroute you provided, the only sign of a potential problem appears to be on Level3's network in Dallas. I should also clarify that we did not fix the issue your user was experiencing a few days ago, as we were unable to duplicate the problem.
I just ran a test from just-ping.com to your web server, and it looks very good from over 30 locations worldwide.
Thank you,
Network Operations Layered Technologies
Tru: I hope you will reopen the Trouble Ticket with Layered Tech, again! With the mtr -c 10 -r centos.org command Rainer used from Germany (which I used from here and posted results in previous message) and the traceroute Per did from Sweden, with my extremely limited knowledge, I believe the problem is *not* on the Level3 network in Dallas, but in the Layered Tech DC in Dallas. Thank you! Lanny
Lanny Marcus wrote:
Tru: I hope you will reopen the Trouble Ticket with Layered Tech, again! With the mtr -c 10 -r centos.org command Rainer used from Germany (which I used from here and posted results in previous message) and the traceroute Per did from Sweden, with my extremely limited knowledge, I believe the problem is *not* on the Level3 network in Dallas, but in the Layered Tech DC in Dallas. Thank you!
You seem to be the only one having that problem - are you sure that it is *not* on your side?
Ralph
On 3/20/09, Ralph Angenendt ra+centos@br-online.de wrote:
Lanny Marcus wrote:
Tru: I hope you will reopen the Trouble Ticket with Layered Tech, again! With the mtr -c 10 -r centos.org command Rainer used from Germany (which I used from here and posted results in previous message) and the traceroute Per did from Sweden, with my extremely limited knowledge, I believe the problem is *not* on the Level3 network in Dallas, but in the Layered Tech DC in Dallas. Thank you!
You seem to be the only one having that problem - are you sure that it is *not* on your side?
Ralph: A few nights ago, John Pierce found things not working properly, when he pinged centos.org from one of his servers in San Jose and another John also saw problems. This morning, Rainer in Germany found things awry, when he used the "mtr -c 10 -r centos.org" command and Per did a traceroute from Sweden, that as I recall, died, within the Layered Tech DC. You are correct in that I am the only one reporting http problems. The Support Tech in Layered Tech reported that he thought there is a problem with Level3 in Dallas, but my impression (and I think that of other people in the list, who are far more knowledgeable than I am) is that the problem is within the Layered Tech DC. Lanny
On Mon, Mar 16, 2009 at 12:17 PM, Lanny Marcus lmmailinglists@gmail.com wrote:
The web site for CentOS is not responding. I can ping it OK, but the web site is not responding to http requests from my browser. If the webmaster should happen to read this...
I know it is working for everyone else. Below is the traceroute. The very slow connection is due to us going over satellite (or via submarine cable?), from Colombia into Miami. I suspect there is a router down or some other problem they will eventually discover? I can load pages from foxnews.com mobile-review.com youtube.com and my 2 sites on servers in CT, without any problems.
[lanny@dell2400 ~]$ traceroute centos.org traceroute to centos.org (72.232.194.162), 30 hops max, 40 byte packets 1 ipcop233 (192.168.10.1) 0.387 ms 0.370 ms 0.394 ms 2 192.168.1.1 (192.168.1.1) 1.289 ms 1.301 ms 1.525 ms 3 dsl-emcali-190.1.240.1.emcali.net.co (190.1.240.1) 18.124 ms 27.454 ms 28.734 ms 4 172.16.1.3 (172.16.1.3) 22.938 ms 16.108 ms 21.446 ms 5 190.90.2.25 (190.90.2.25) 63.484 ms 64.420 ms 63.804 ms 6 so-4-2-3-nmi-core01.nwnnetwork.net (63.245.40.157) 123.859 ms 124.120 ms 181.105 ms 7 ge-1-1-0-nmi-core02.columbus-networks.com (63.245.5.0) 136.454 ms 137.090 ms 137.423 ms 8 * * * 9 * * * 10 ae-2.ebr1.Dallas1.Level3.net (4.69.140.133) 144.717 ms 109.365 ms 107.428 ms 11 ae-71-71.csw2.Dallas1.Level3.net (4.69.136.126) 113.921 ms ae-81-81.csw3.Dallas1.Level3.net (4.69.136.130) 103.768 ms ae-91-91.csw4.Dallas1.Level3.net (4.69.136.134) 112.911 ms 12 ae-22-79.car2.Dallas1.Level3.net (4.68.19.68) 107.676 ms ae-42-99.car2.Dallas1.Level3.net (4.68.19.196) 107.077 ms ae-32-89.car2.Dallas1.Level3.net (4.68.19.132) 107.756 ms 13 DATABANK-HO.car2.Dallas1.Level3.net (4.71.170.2) 108.020 ms 108.180 ms 125.328 ms 14 aw_cw_10g.databank.com (63.164.96.54) 125.352 ms ae_cw_10g.databank.com (63.164.96.62) 125.568 ms 123.029 ms 15 pod22c_ae.layeredtech.com (63.164.96.202) 126.035 ms pod22a_aw.layeredtech.com (63.164.96.242) 126.307 ms pod22c_ae.layeredtech.com (63.164.96.202) 126.186 ms 16 162.194.232.72.static.reverse.ltdomains.com (72.232.194.162) 128.613 ms !X 128.493 ms !X 129.024 ms !X [lanny@dell2400 ~]$
Lanny Marcus wrote:
On Mon, Mar 16, 2009 at 12:17 PM, Lanny Marcus lmmailinglists@gmail.com wrote:
The web site for CentOS is not responding. I can ping it OK, but the web site is not responding to http requests from my browser. If the webmaster should happen to read this...
I know it is working for everyone else. Below is the traceroute. The very slow connection is due to us going over satellite (or via submarine cable?), from Colombia into Miami.
its not satellite, or there'd be a hop in the middle that added about 500mS to all further points. speed of light up to geostationary orbit (about 22800 miles, +/- based on the lat/long of the earth stations, and the longitude of the satellite)) and back twice, kinda hard to beat that, and the main reason satellite internet isn't very popular.
*per google, (22800 miles * 4) / c = 489.58 milliseconds*
(* 4 because your packets woudl go up to orbit and back down, then the response has to go up and back again.)
On Mon, Mar 16, 2009 at 4:36 PM, John R Pierce pierce@hogranch.com wrote:
Lanny Marcus wrote:
On Mon, Mar 16, 2009 at 12:17 PM, Lanny Marcus lmmailinglists@gmail.com wrote:
The web site for CentOS is not responding. I can ping it OK, but the web site is not responding to http requests from my browser. If the webmaster should happen to read this...
I know it is working for everyone else. Below is the traceroute. The very slow connection is due to us going over satellite (or via submarine cable?), from Colombia into Miami.
its not satellite, or there'd be a hop in the middle that added about 500mS to all further points. speed of light up to geostationary orbit (about 22800 miles, +/- based on the lat/long of the earth stations, and the longitude of the satellite)) and back twice, kinda hard to beat that, and the main reason satellite internet isn't very popular.
*per google, (22800 miles * 4) / c = 489.58 milliseconds*
(* 4 because your packets woudl go up to orbit and back down, then the response has to go up and back again.)
That's good to know. I seem to recall, years ago, they were installing a Submarine cable to the USA.
Do you see anything in the traceroute that shows where the problem is? It still won't load for me. I can see when it does the DNS lookup and after that the browser status is "Waiting for www.centos.org"
On Mon, 2009-03-16 at 17:04 -0500, Lanny Marcus wrote:
On Mon, Mar 16, 2009 at 4:36 PM, John R Pierce pierce@hogranch.com wrote:
Lanny Marcus wrote:
On Mon, Mar 16, 2009 at 12:17 PM, Lanny Marcus lmmailinglists@gmail.com wrote:
The web site for CentOS is not responding. I can ping it OK, but the web site is not responding to http requests from my browser. If the webmaster should happen to read this...
I know it is working for everyone else. Below is the traceroute. The very slow connection is due to us going over satellite (or via submarine cable?), from Colombia into Miami.
-----------
Something fishey going on here! You can send an email to @centos.org but not www?? Dump all your browser cache out. Appears the traceroute made it.
JohnStanley
On Mon, Mar 16, 2009 at 5:45 PM, JohnS jses27@gmail.com wrote:
On Mon, 2009-03-16 at 17:04 -0500, Lanny Marcus wrote:
On Mon, Mar 16, 2009 at 4:36 PM, John R Pierce pierce@hogranch.com wrote:
Lanny Marcus wrote:
On Mon, Mar 16, 2009 at 12:17 PM, Lanny Marcus lmmailinglists@gmail.com wrote:
The web site for CentOS is not responding. I can ping it OK, but the web site is not responding to http requests from my browser. If the webmaster should happen to read this...
I know it is working for everyone else. Below is the traceroute. The very slow connection is due to us going over satellite (or via submarine cable?), from Colombia into Miami.
Something fishey going on here! You can send an email to @centos.org but not www?? Dump all your browser cache out. Appears the traceroute made it.
John: I think Marcelo in Argentina hit the nail on the head. Apparently, either layeredtech in Dallas or CentOS is now blocking the IP I am using today, or has begun blocking all IP addresses from Colombia, so I cannot get to the centos web site anymore. Firefox Cache is cleared, automatically, each time I close the browser. We do not pay extra for a Dedicated IP from our ISP, so I am starting to wonder if the one we are using today has been blocked by layeredtech. Or, all of them... Lanny
On Mon, Mar 16, 2009 at 6:08 PM, Lanny Marcus lmmailinglists@gmail.comwrote:
On Mon, Mar 16, 2009 at 5:45 PM, JohnS jses27@gmail.com wrote:
On Mon, 2009-03-16 at 17:04 -0500, Lanny Marcus wrote:
On Mon, Mar 16, 2009 at 4:36 PM, John R Pierce pierce@hogranch.com
wrote:
Lanny Marcus wrote:
On Mon, Mar 16, 2009 at 12:17 PM, Lanny Marcus <
lmmailinglists@gmail.com> wrote:
The web site for CentOS is not responding. I can ping it OK, but the web site is not responding to http requests from my browser. If the webmaster should happen to read this...
I know it is working for everyone else. Below is the traceroute. The very slow connection is due to us going over satellite (or via submarine cable?), from Colombia into Miami.
Something fishey going on here! You can send an email to @centos.org but not www?? Dump all your browser cache out. Appears the traceroute made it.
John: I think Marcelo in Argentina hit the nail on the head. Apparently, either layeredtech in Dallas or CentOS is now blocking the IP I am using today, or has begun blocking all IP addresses from Colombia, so I cannot get to the centos web site anymore. Firefox Cache is cleared, automatically, each time I close the browser. We do not pay extra for a Dedicated IP from our ISP, so I am starting to wonder if the one we are using today has been blocked by layeredtech. Or, all of them... Lanny _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Try restarting your router/modem in order to adquire a new IP address from the ISP's pool. maybe that could work
2009/3/16 Victor Padro vpadro@gmail.com:
On Mon, Mar 16, 2009 at 6:08 PM, Lanny Marcus lmmailinglists@gmail.com
<snip>
> The web site for CentOS is not responding. I can ping it OK, but > the > web site is not responding to http requests from my browser. If the > webmaster should happen to read this... > I know it is working for everyone else. Below is the traceroute. The very slow connection is due to us going over satellite (or via submarine cable?), from Colombia into Miami.
<snip>
Try restarting your router/modem in order to adquire a new IP address from the ISP's pool. maybe that could work
Yuk. That is what I did for a long time, so I could use the SMTP on our web sites. Frequently, the IP's I'd get from our ISP were listed with Spamhaus. I gave up and started using Gmail... I will try that, if my wife isn't surfing at this time. It could well be that layeredtech has blocked the IP I got this morning, from the server centos.org uses there.
On Tue, 2009-03-17 at 11:20 +0100, Ralph Angenendt wrote:
JohnS wrote:
Something fishey going on here! You can send an email to @centos.org but not www??
Sure. The mailserver is nowhere near to the web server. Okay, same continent, but completely different DC.
Ralph
Ahh, ok I see now :-)
JohnStanley
Lanny Marcus wrote:
That's good to know. I seem to recall, years ago, they were installing a Submarine cable to the USA.
Do you see anything in the traceroute that shows where the problem is? It still won't load for me. I can see when it does the DNS lookup and after that the browser status is "Waiting for www.centos.org"
not really. FWIW, here's -my- traceroute from somewhere in northern california to the server...
$ traceroute www.centos.org traceroute to www.centos.org (72.232.194.162), 30 hops max, 40 byte packets 1 ge-0-0-1-414.er2.sjc1.got.net (207.111.214.242) 3.254 ms 3.159 ms 3.131 ms 2 ge-1-0-10.cr1.sjc1.got.net (207.111.208.49) 27.850 ms 28.121 ms 28.106 ms 3 173.ge-1-3-0.mpr1.sjc2.us.above.net (64.124.193.34) 4.266 ms 4.264 ms 4.245 ms 4 xe-1-1-0.er1.sjc7.us.above.net (64.125.27.89) 4.226 ms 4.208 ms 4.185 ms 5 above-level3.sjc7.us.above.net (64.125.13.242) 19.989 ms 19.988 ms 19.968 ms 6 vlan99.csw4.SanJose1.Level3.net (4.68.18.254) 13.772 ms vlan79.csw2.SanJose1.Level3.net (4.68.18.126) 16.030 ms vlan99.csw4.SanJose1.Level3.net (4.68.18.254) 4.494 ms 7 ae-83-83.ebr3.SanJose1.Level3.net (4.69.134.233) 12.551 ms ae-63-63.ebr3.SanJose1.Level3.net (4.69.134.225) 12.524 ms ae-83-83.ebr3.SanJose1.Level3.net (4.69.134.233) 12.477 ms 8 ae-2.ebr3.LosAngeles1.Level3.net (4.69.132.10) 21.245 ms 21.348 ms 21.159 ms 9 ae-73-73.csw2.LosAngeles1.Level3.net (4.69.137.38) 24.868 ms ae-63-63.csw1.LosAngeles1.Level3.net (4.69.137.34) 12.927 ms ae-83-83.csw3.LosAngeles1.Level3.net (4.69.137.42) 21.287 ms 10 ae-82-82.ebr2.LosAngeles1.Level3.net (4.69.137.25) 22.462 ms ae-72-72.ebr2.LosAngeles1.Level3.net (4.69.137.21) 22.445 ms ae-62-62.ebr2.LosAngeles1.Level3.net (4.69.137.17) 22.431 ms 11 * * ae-3.ebr3.Dallas1.Level3.net (4.69.132.78) 46.555 ms 12 ae-83-83.csw3.Dallas1.Level3.net (4.69.136.162) 60.050 ms ae-63-63.csw1.Dallas1.Level3.net (4.69.136.154) 52.497 ms ae-83-83.csw3.Dallas1.Level3.net (4.69.136.162) 60.547 ms 13 ae-42-99.car2.Dallas1.Level3.net (4.68.19.196) 47.712 ms ae-32-89.car2.Dallas1.Level3.net (4.68.19.132) 48.047 ms 46.451 ms 14 DATABANK-HO.car2.Dallas1.Level3.net (4.71.170.2) 47.981 ms 47.943 ms 47.939 ms 15 aw_cw_10g.databank.com (63.164.96.54) 46.343 ms 46.067 ms ae_cw_10g.databank.com (63.164.96.62) 46.301 ms 16 pod22e_aw.layeredtech.com (63.164.96.178) 48.343 ms pod22c_ae.layeredtech.com (63.164.96.202) 48.309 ms pod22a_aw.layeredtech.com (63.164.96.242) 48.306 ms 17 162.194.232.72.static.reverse.ltdomains.com (72.232.194.162) 48.485 ms !X 49.041 ms !X 48.281 ms !X
and when I run this from a different norcal ISP,
$ traceroute -I www.centos.org traceroute to www.centos.org (72.232.194.162), 30 hops max, 38 byte packets 1 75-101-82-1.static.dsltransport.net (75.101.82.1) 8.628 ms 8.502 ms 8.819 ms 2 128.at-X-X-X.gw3.200p-sf.sonic.net (208.106.96.201) 8.833 ms 9.038 ms 9.060 ms 3 0.as0.gw4.200p-sf.sonic.net (64.142.0.226) 8.844 ms 9.040 ms 8.797 ms 4 ge-6-22.car1.SanFrancisco1.Level3.net (4.53.128.97) 9.374 ms 9.272 ms 9.305 ms 5 ae-2-4.bar1.SanFrancisco1.Level3.net (4.69.133.150) 9.855 ms 9.420 ms 9.792 ms 6 ae-0-11.bar2.SanFrancisco1.Level3.net (4.69.140.146) 9.347 ms 9.462 ms 8.821 ms 7 ae-6-6.ebr2.SanJose1.Level3.net (4.69.140.154) 17.496 ms 16.811 ms 17.672 ms 8 ae-72-72.csw2.SanJose1.Level3.net (4.69.134.214) 18.713 ms 22.376 ms 17.419 ms 9 ae-73-73.ebr3.SanJose1.Level3.net (4.69.134.229) 22.111 ms 22.142 ms 17.930 ms 10 ae-2.ebr3.LosAngeles1.Level3.net (4.69.132.10) 18.421 ms 18.059 ms 17.632 ms 11 ae-73-73.csw2.LosAngeles1.Level3.net (4.69.137.38) 25.817 ms 18.322 ms 18.100 ms 12 ae-72-72.ebr2.LosAngeles1.Level3.net (4.69.137.21) 20.676 ms 18.322 ms 18.452 ms 13 ae-3.ebr3.Dallas1.Level3.net (4.69.132.78) 57.892 ms 53.068 ms 53.730 ms 14 ae-93-93.csw4.Dallas1.Level3.net (4.69.136.166) 54.260 ms 52.844 ms 53.874 ms 15 ae-42-99.car2.Dallas1.Level3.net (4.68.19.196) 51.354 ms 51.205 ms 50.922 ms 16 DATABANK-HO.car2.Dallas1.Level3.net (4.71.170.2) 50.216 ms 50.984 ms 50.895 ms 17 aw_cw_10g.databank.com (63.164.96.54) 51.217 ms 50.102 ms 51.184 ms 18 pod22a_aw.layeredtech.com (63.164.96.242) 50.703 ms 50.165 ms 50.450 ms 19 * * * 20 162.194.232.72.static.reverse.ltdomains.com (72.232.194.162) 51.661 ms 50.419 ms 50.880 ms
On Mon, Mar 16, 2009 at 7:50 PM, John R Pierce pierce@hogranch.com wrote:
Lanny Marcus wrote:
[...]
17 162.194.232.72.static.reverse.ltdomains.com (72.232.194.162) 48.485 ms !X 49.041 ms !X 48.281 ms !X
As man traceroute states, "!X" is communication administratively prohibited... I think they're blocking you somehow...
On Mon, Mar 16, 2009 at 6:09 PM, Marcelo Roccasalva marcelo-centos@irrigacion.gov.ar wrote:
On Mon, Mar 16, 2009 at 7:50 PM, John R Pierce pierce@hogranch.com wrote:
Lanny Marcus wrote:
[...]
17 162.194.232.72.static.reverse.ltdomains.com (72.232.194.162) 48.485 ms !X 49.041 ms !X 48.281 ms !X
As man traceroute states, "!X" is communication administratively prohibited... I think they're blocking you somehow...
16 162.194.232.72.static.reverse.ltdomains.com (72.232.194.162) 136.603 ms !X 136.185 ms !X 133.898 ms !X
Marcelo: That is *very* interesting. ¡Gracias! This is probably the first time in a week or two that I have tried to surf CentOS.org and I've *never* had this problem before. If they are now blocking me, because my IP address is in Colombia, I have no idea why they would do that. That's the last hop in the traceroute to centos.org I wonder if that is something layeredtech.com took it upon themselves to implement against those of us in Colombia? Lanny
On Mon, 2009-03-16 at 20:09 -0300, Marcelo Roccasalva wrote:
On Mon, Mar 16, 2009 at 7:50 PM, John R Pierce pierce@hogranch.com wrote:
Lanny Marcus wrote:
[...]
17 162.194.232.72.static.reverse.ltdomains.com (72.232.194.162) 48.485 ms !X 49.041 ms !X 48.281 ms !X
As man traceroute states, "!X" is communication administratively prohibited... I think they're blocking you somehow...
---- That's not true or the case in fact that would be wrong. Here is my traceroute:
13 162.194.232.72.static.reverse.ltdomains.com (72.232.194.162) 217.811 ms !X 254.820 \ ms !X 267.841 ms !X
In which the whole @centos.org domain is accessable to me! So that man page was read or understood wrong!
JohnStanley
On Mon, Mar 16, 2009 at 5:50 PM, John R Pierce pierce@hogranch.com wrote:
Lanny Marcus wrote:
That's good to know. I seem to recall, years ago, they were installing a Submarine cable to the USA.
Do you see anything in the traceroute that shows where the problem is? It still won't load for me. I can see when it does the DNS lookup and after that the browser status is "Waiting for www.centos.org"
not really. FWIW, here's -my- traceroute from somewhere in northern california to the server...
From San Jose you are also going into layeredtech via level3 as I am,
once I get to Miami. Rick wrote me, off list, to try traceroute with -T and here's what I got:
[root@dell2400 ~]# traceroute -T centos.org traceroute to centos.org (72.232.194.162), 30 hops max, 40 byte packets 1 ipcop233 (192.168.10.1) 0.608 ms 0.533 ms 0.534 ms 2 192.168.1.1 (192.168.1.1) 1.596 ms 2.022 ms 1.117 ms 3 162.194.232.72.static.reverse.ltdomains.com (72.232.194.162) 102.337 ms 101.890 ms 20.552 ms [root@dell2400 ~]
Had a suggestion I try to get a different IP from our ISP and if my wife isn't surfing now, I will try that.. Until today, I have only run into one (1) web site that won't let me on, because my IP is in Colombia. CentOS possibly has become the 2nd one. I hope not!
Lanny Marcus wrote:
Had a suggestion I try to get a different IP from our ISP and if my wife isn't surfing now, I will try that.. Until today, I have only run into one (1) web site that won't let me on, because my IP is in Colombia. CentOS possibly has become the 2nd one. I hope not!
I did some more pinging later on from one of my servers in San Jose, California, and saw some signs of route instability at layeredwhazza where the next hop after what should have been hte next-to-last one was coming from various routers all going 'no route to destination'
Lanny Marcus wrote:
Had a suggestion I try to get a different IP from our ISP and if my wife isn't surfing now, I will try that.. Until today, I have only run into one (1) web site that won't let me on, because my IP is in Colombia. CentOS possibly has become the 2nd one. I hope not!
-----
One thing you could try is if you have a friend that lives there in Columbia is to try and access it from there computer. They just need to have the same Internet Service Provider. Another way would be through a Proxy Server that is anonymous. But doing another trace to layeredtech and ltdomains shows bad hops also from my end just like John Pierces shows. But still I can access it by WWW.
JohnStanley
On Tue, Mar 17, 2009 at 1:05 AM, JohnS jses27@gmail.com wrote:
Lanny Marcus wrote:
Had a suggestion I try to get a different IP from our ISP and if my wife isn't surfing now, I will try that.. Until today, I have only run into one (1) web site that won't let me on, because my IP is in Colombia. CentOS possibly has become the 2nd one. I hope not!
One thing you could try is if you have a friend that lives there in Columbia is to try and access it from there computer. They just need to have the same Internet Service Provider. Another way would be through a Proxy Server that is anonymous. But doing another trace to layeredtech and ltdomains shows bad hops also from my end just like John Pierces shows. But still I can access it by WWW.
John: Thank you for also confirming there is a network connectivity problem in or near layered tech in Dallas! Much appreciated! I will try to find a free anonymous proxy service that can load the HTML pages, until they cure this glitch. I do not see on the layeredtech.com contact page an address for network connectivity problems. I am going to send an email to support AT layeredtech DOT com and hopefully that will be Forwarded to a Network Tech there who can look into what you and John Pierce found. Lanny
On Tue, Mar 17, 2009 at 3:38 AM, Lanny Marcus lmmailinglists@gmail.com wrote: <snip>
Proxy Server that is anonymous. But doing another trace to layeredtech and ltdomains shows bad hops also from my end just like John Pierces shows. But still I can access it by WWW.
<snip>
I sent an email to support AT layeredtech DOT com got their auto responder and 5 minutes later Kevin wrote that they need the Server ID and Client ID.
I replied to Kevin that I'm only an end user at home and that I will post here and ask one of the CentOS team with Administrative privileges for that server to contact them for support.
Sorry for the double post(s) in this thread. gmail died a couple of times, when I was trying to send email.....
Thank you, to everyone who replied to this thread. Especially to the two John's, who continued pinging centos.org and discovered there is a problem, in or near the Layered Tech NOC in Dallas.
On Tue, Mar 17, 2009 at 04:23:47AM -0500, Lanny Marcus wrote:
On Tue, Mar 17, 2009 at 3:38 AM, Lanny Marcus lmmailinglists@gmail.com wrote:
<snip> >> Proxy Server that is anonymous. But doing another trace to layeredtech >> and ltdomains shows bad hops also from my end just like John Pierces >> shows. But still I can access it by WWW. <snip>
I sent an email to support AT layeredtech DOT com got their auto responder and 5 minutes later Kevin wrote that they need the Server ID and Client ID.
I replied to Kevin that I'm only an end user at home and that I will post here and ask one of the CentOS team with Administrative privileges for that server to contact them for support.
I have opened a ticket pointing back to this thread at the hoster.
Thanks
Tru
2009/3/17 Tru Huynh tru@centos.org:
On Tue, Mar 17, 2009 at 04:23:47AM -0500, Lanny Marcus wrote:
On Tue, Mar 17, 2009 at 3:38 AM, Lanny Marcus lmmailinglists@gmail.com wrote:
<snip> >> Proxy Server that is anonymous. But doing another trace to layeredtech >> and ltdomains shows bad hops also from my end just like John Pierces >> shows. But still I can access it by WWW. <snip>
I sent an email to support AT layeredtech DOT com got their auto responder and 5 minutes later Kevin wrote that they need the Server ID and Client ID.
I replied to Kevin that I'm only an end user at home and that I will post here and ask one of the CentOS team with Administrative privileges for that server to contact them for support.
I have opened a ticket pointing back to this thread at the hoster.
Thanks
Tru: THANK YOU! I wrote to Russ, off line, about 4 hours ago. My impression is that the people at Layered Tech are very professional. I'm sure they will eliminate this problem, ASAP. Lanny
On Tue, Mar 17, 2009 at 8:52 AM, Lanny Marcus lmmailinglists@gmail.com wrote:
2009/3/17 Tru Huynh tru@centos.org:
<snip> e and that I will
post here and ask one of the CentOS team with Administrative privileges for that server to contact them for support.
I have opened a ticket pointing back to this thread at the hoster.
Thanks
Tru: THANK YOU! I wrote to Russ, off line, about 4 hours ago. My impression is that the people at Layered Tech are very professional. I'm sure they will eliminate this problem, ASAP. Lanny
They did. I can load the web pages from centos.org again. :-) Thanks to everyone on the list who helped and also to the person at Layered Tech who eliminated the glitch!
On Tue, Mar 17, 2009 at 09:36:28AM -0500, Lanny Marcus wrote:
They did. I can load the web pages from centos.org again. :-) Thanks to everyone on the list who helped and also to the person at Layered Tech who eliminated the glitch!
<quoting them (Network Operations, Layered Technologies) > ... It looks like there may be a probelm with an offsite (non LT) router. I will have networking investigate and get back to you as soon as possible. ... </quote>
Then a few minutes later :)
<quote> Dear Customer:
I've read through the thread on your message boards, and don't see anything that jumps out as a problem on the traceroutes. The "Administrative block" results you are seeing are likely due to default traceroute behavior on *nix systems is to use UDP as the sending protocol. In order to prevent UDP flood attacks against our network, our routers and switches do not respond to UDP requests. The best way to run a traceroute to a server in our data centers is to force traceroute to use ICMP instead of UDP. On most *nix systems, this can be done by using the "-I" flag.
In answer to other speculation on the thread, we don't block any IPs from our network as a whole. Our business is to allow you to make your server available to anyone and everyone you want to have access to it. Therefore, we only block IPs if a customer has a hardware firewall that we manage for them, and we only block it utilizing that customer's firewall and at that customer's request. Therefore, any blocks that are put in place do not affect other customers' servers.
I hope this clarifies some of the issues, although it doesn't resolve the problem the user in Colombia is having reaching your site. Another test he could perform would just be to ping your server's IP and see if he gets a response -- if he does, then the routing and switching is working correctly.
Thank you,
Network Operations Layered Technologies </quote>
2009/3/17 Tru Huynh tru@centos.org:
On Tue, Mar 17, 2009 at 09:36:28AM -0500, Lanny Marcus wrote:
They did. I can load the web pages from centos.org again. :-) Thanks to everyone on the list who helped and also to the person at Layered Tech who eliminated the glitch!
<quoting them (Network Operations, Layered Technologies) > ... It looks like there may be a probelm with an offsite (non LT) router. I will have networking investigate and get back to you as soon as possible. ...
</quote>
Then a few minutes later :)
<snip.
Tru: Thank you for posting all of that. My first impression yesterday was that maybe there was a problem with Level3, somewhere between Miami and Dallas. I think John Pierce thought last night that it was a router very close to the CentOS server. Level3 is normally very good. One of my web sites in CT uses Level3 from Miami. At this time, centos.org is loading OK for me. The strange thing for me is that everyone who responded was able to load centos.org OK, so there must be something about our connection that caused it to fail for me. If LT notifies you what the problem was, I'm not the only one who is curious. I can ping the IP without any problem, but I was also able to ping the Domain Name without any problem. The thing I did not report yesterday was that the ping times were varying, a lot. This morning, and now, within about 5 ms, which is the same as for my web sites and what I consider "normal". Many thanks for contacting LT! Lanny
On Tue, Mar 17, 2009 at 1:05 AM, JohnS jses27@gmail.com wrote:
Lanny Marcus wrote:
Had a suggestion I try to get a different IP from our ISP and if my wife isn't surfing now, I will try that.. Until today, I have only run into one (1) web site that won't let me on, because my IP is in Colombia. CentOS possibly has become the 2nd one. I hope not!
One thing you could try is if you have a friend that lives there in Columbia is to try and access it from there computer. They just need to have the same Internet Service Provider. Another way would be through a Proxy Server that is anonymous. But doing another trace to layeredtech and ltdomains shows bad hops also from my end just like John Pierces shows. But still I can access it by WWW.
John: Thank you for also confirming there is a network connectivity problem in or near layered tech in Dallas! Much appreciated! I will try to find a free anonymous proxy service that can load the HTML pages, until they cure this glitch. I do not see on the layeredtech.com contact page an address for network connectivity problems. I am going to send an email to support AT layeredtech DOT com and hopefully that will be Forwarded to a Network Tech there who can look into what you and John Pierce found. Lanny
On Mon, Mar 16, 2009 at 8:40 PM, John R Pierce pierce@hogranch.com wrote:
Lanny Marcus wrote:
Had a suggestion I try to get a different IP from our ISP and if my wife isn't surfing now, I will try that.. Until today, I have only run into one (1) web site that won't let me on, because my IP is in Colombia. CentOS possibly has become the 2nd one. I hope not!
I did some more pinging later on from one of my servers in San Jose, California, and saw some signs of route instability at layeredwhazza where the next hop after what should have been hte next-to-last one was coming from various routers all going 'no route to destination'
John: Thank you for doing that additional testing! I came in here just before 3 A.M. and powered up the ADSL modem, IPCop box, etc. Probably have a different IP than I had yesterday. Still can't load pages from centos.org
One thing that I noticed yesterday, when I pinged centos.org was that the ping times were varying a lot. I did not mention that yesterday. However, at this time, the ping times for centos.org from here are about the same as they are for my 2 web sites, which are on servers in different NOCs in CT. Within approximately 5 ms. I also found that the site I could not load last year is now loading, without any problem. Sounds like you found a problem they can fix and that they are not blocking IPs from Colombia. :-) Lanny
On Mon, Mar 16, 2009 at 8:40 PM, John R Pierce pierce@hogranch.com wrote: <snip>
I did some more pinging later on from one of my servers in San Jose, California, and saw some signs of route instability at layeredwhazza where the next hop after what should have been hte next-to-last one was coming from various routers all going 'no route to destination'
I want to bring John's words above back into the thread again, because this is one of the most important messages in the thread.
Lanny Marcus wrote:
On Mon, Mar 16, 2009 at 8:40 PM, John R Pierce pierce@hogranch.com wrote:
<snip>
I did some more pinging later on from one of my servers in San Jose, California, and saw some signs of route instability at layeredwhazza where the next hop after what should have been hte next-to-last one was coming from various routers all going 'no route to destination'
I want to bring John's words above back into the thread again, because this is one of the most important messages in the thread.
I wouldn't put too much faith in that, however. i only saw this once.
On Fri, Mar 20, 2009 at 2:48 PM, John R Pierce pierce@hogranch.com wrote:
On Mon, Mar 16, 2009 at 8:40 PM, John R Pierce pierce@hogranch.com wrote:
<snip> > I did some more pinging later on from one of my servers in San Jose, > California, and saw some signs of route instability at layeredwhazza > where the next hop after what should have been hte next-to-last one was > coming from various routers all going 'no route to destination'
<snip>
I wouldn't put too much faith in that, however. i only saw this once.
John: I think Per from Sweden this morning (USA time) found that his traceroute to the server died, within Layered Tech. After Tru filed the Trouble Ticket with LT the other day, it started working again for me, but now it's dead again down here. The LT Tech replied he thought there was a problem on Level3 in Dallas, but my bet is the problem is within the LT DC. Ping times from here are higher than they were a couple of days ago, but faster than pings to my 2 sites in the USA (on shared servers) and now they are very close. The other day, there was a lot of variance in ping times to centos.org Intermittent problems are always the worst to eliminate.... Lanny
On 3/16/09, Lanny Marcus lmmailinglists@gmail.com wrote:
The web site for CentOS is not responding. I can ping it OK, but the web site is not responding to http requests from my browser. If the webmaster should happen to read this...
They (Layered Tech or Level3) fixed the problem the other day. I have been checking, periodically, to see if I can get to centos.org and the web pages have been loading fine. However, I just checked again and now I have the same problem again: (a) I can ping centos.org OK, but the ping times are higher than when it works OK (b) Firefox status shows "Waiting for www.centos.org" after the DNS lookup
If the problem is on Level3, our route is on Level3, from Miami into Dallas.
8 xe-8-0-0.edge2.Miami1.Level3.net (4.59.84.5) 94.386 ms 96.005 ms 98.375 ms 9 ae-31-51.ebr1.Miami1.Level3.net (4.69.138.94) 103.582 ms 103.800 ms 105.502 ms 10 ae-2.ebr1.Dallas1.Level3.net (4.69.140.133) 143.348 ms 143.496 ms 143.799 ms 11 ae-61-61.csw1.Dallas1.Level3.net (4.69.136.122) 150.643 ms * * 12 * * * 13 * DATABANK-HO.car2.Dallas1.Level3.net (4.71.170.2) 115.438 ms 119.708 ms 14 ae_cw_10g.databank.com (63.164.96.62) 122.067 ms aw_cw_10g.databank.com (63.164.96.54) 127.471 ms ae_cw_10g.databank.com (63.164.96.62) 126.691 ms 15 pod22c_ae.layeredtech.com (63.164.96.202) 130.134 ms pod22a_aw.layeredtech.com (63.164.96.242) 132.261 ms 135.707 ms 16 162.194.232.72.static.reverse.ltdomains.com (72.232.194.162) 114.324 ms !X 117.706 ms !X 118.991 ms !X
On Mon, Mar 16, 2009 at 12:17 PM, Lanny Marcus lmmailinglists@gmail.com wrote:
The web site for CentOS is not responding. I can ping it OK, but the web site is not responding to http requests from my browser. If the webmaster should happen to read this...
As I posted late yesterday afternoon, the problem went away a few days ago, but then began again, yesterday. I can ping the server and traceroute to it, but no http for me from centos.org again. Tru, if you read this, I hope you will reopen the Trouble Ticket at Layered Tech, again. If they believe the problem is not theirs, hopefully they will open a Trouble Ticket with Level3. Traceroute I just made is below. Thank you! Lanny
[lanny@dell2400 ~]$ traceroute centos.org traceroute to centos.org (72.232.194.162), 30 hops max, 40 byte packets 1 ipcop233 (192.168.10.1) 0.525 ms 0.518 ms 0.530 ms 2 192.168.1.1 (192.168.1.1) 2.100 ms 2.958 ms 2.927 ms 3 dsl-emcali-190.1.240.1.emcali.net.co (190.1.240.1) 23.220 ms 25.265 ms 27.106 ms 4 172.16.1.3 (172.16.1.3) 29.382 ms 31.392 ms 33.774 ms 5 190.90.2.25 (190.90.2.25) 68.564 ms 68.975 ms 69.220 ms 6 so-4-2-3-nmi-core01.nwnnetwork.net (63.245.40.157) 105.957 ms * * 7 * ge-1-1-0-nmi-core02.columbus-networks.com (63.245.5.0) 78.319 ms 76.043 ms 8 xe-8-0-0.edge2.Miami1.Level3.net (4.59.84.5) 89.541 ms 92.189 ms 94.493 ms 9 ae-31-51.ebr1.Miami1.Level3.net (4.69.138.94) 99.756 ms 100.047 ms 101.620 ms 10 ae-2.ebr1.Dallas1.Level3.net (4.69.140.133) 144.085 ms 144.384 ms 144.732 ms 11 ae-71-71.csw2.Dallas1.Level3.net (4.69.136.126) 147.074 ms * * 12 * * ae-42-99.car2.Dallas1.Level3.net (4.68.19.196) 137.429 ms 13 DATABANK-HO.car2.Dallas1.Level3.net (4.71.170.2) 140.261 ms 114.191 ms 118.113 ms 14 ae_cw_10g.databank.com (63.164.96.62) 120.519 ms 124.321 ms aw_cw_10g.databank.com (63.164.96.54) 128.517 ms 15 pod22e_aw.layeredtech.com (63.164.96.178) 128.776 ms 130.954 ms pod22h_ae.layeredtech.com (63.164.96.162) 132.805 ms 16 162.194.232.72.static.reverse.ltdomains.com (72.232.194.162) 114.030 ms !X 116.744 ms !X 116.070 ms !X [lanny@dell2400 ~]$
Lanny Marcus schrieb:
On Mon, Mar 16, 2009 at 12:17 PM, Lanny Marcus lmmailinglists@gmail.com wrote:
The web site for CentOS is not responding. I can ping it OK, but the web site is not responding to http requests from my browser. If the webmaster should happen to read this...
As I posted late yesterday afternoon, the problem went away a few days ago, but then began again, yesterday. I can ping the server and traceroute to it, but no http for me from centos.org again. Tru, if you read this, I hope you will reopen the Trouble Ticket at Layered Tech, again. If they believe the problem is not theirs, hopefully they will open a Trouble Ticket with Level3. Traceroute I just made is below. Thank you! Lanny
[lanny@dell2400 ~]$ traceroute centos.org traceroute to centos.org (72.232.194.162), 30 hops max, 40 byte packets 1 ipcop233 (192.168.10.1) 0.525 ms 0.518 ms 0.530 ms 2 192.168.1.1 (192.168.1.1) 2.100 ms 2.958 ms 2.927 ms 3 dsl-emcali-190.1.240.1.emcali.net.co (190.1.240.1) 23.220 ms 25.265 ms 27.106 ms 4 172.16.1.3 (172.16.1.3) 29.382 ms 31.392 ms 33.774 ms 5 190.90.2.25 (190.90.2.25) 68.564 ms 68.975 ms 69.220 ms 6 so-4-2-3-nmi-core01.nwnnetwork.net (63.245.40.157) 105.957 ms * * 7 * ge-1-1-0-nmi-core02.columbus-networks.com (63.245.5.0) 78.319 ms 76.043 ms 8 xe-8-0-0.edge2.Miami1.Level3.net (4.59.84.5) 89.541 ms 92.189 ms 94.493 ms 9 ae-31-51.ebr1.Miami1.Level3.net (4.69.138.94) 99.756 ms 100.047 ms 101.620 ms 10 ae-2.ebr1.Dallas1.Level3.net (4.69.140.133) 144.085 ms 144.384 ms 144.732 ms 11 ae-71-71.csw2.Dallas1.Level3.net (4.69.136.126) 147.074 ms * * 12 * * ae-42-99.car2.Dallas1.Level3.net (4.68.19.196) 137.429 ms 13 DATABANK-HO.car2.Dallas1.Level3.net (4.71.170.2) 140.261 ms 114.191 ms 118.113 ms 14 ae_cw_10g.databank.com (63.164.96.62) 120.519 ms 124.321 ms aw_cw_10g.databank.com (63.164.96.54) 128.517 ms 15 pod22e_aw.layeredtech.com (63.164.96.178) 128.776 ms 130.954 ms pod22h_ae.layeredtech.com (63.164.96.162) 132.805 ms 16 162.194.232.72.static.reverse.ltdomains.com (72.232.194.162) 114.030 ms !X 116.744 ms !X 116.070 ms !X [lanny@dell2400 ~]$ _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
FYI: this is what I get from my colo'ed box in Germany:
srv3# traceroute centos.org traceroute to centos.org (72.232.194.162), 64 hops max, 40 byte packets 1 gw.rtr2.colo1.NBG1.content-colo.net (80.190.225.9) 0.376 ms 0.380 ms 0.369 ms 2 p10-91.cr1.NBG1.content-core.net (212.123.123.225) 0.355 ms p10-92.cr2.NBG1.content-core.net (212.123.123.241) 0.375 ms p10-91.cr1.NBG1.content-core.net (212.123.123.225) 0.368 ms 3 te-7-2.edge3.frf1.Level3.net (212.162.19.33) 8.328 ms Tenge1-1-40.cr2.NBG1.content-core.net (212.123.123.122) 0.382 ms te-7-2.edge3.frf1.Level3.net (212.162.19.33) 8.177 ms 4 vlan79.csw2.Frankfurt1.Level3.net (4.68.23.126) 15.194 ms te-7-2.edge3.frf1.Level3.net (212.162.19.33) 8.328 ms vlan79.csw2.Frankfurt1.Level3.net (4.68.23.126) 8.488 ms 5 vlan79.csw2.Frankfurt1.Level3.net (4.68.23.126) 15.031 ms ae-82-82.ebr2.Frankfurt1.Level3.net (4.69.140.25) 8.641 ms vlan79.csw2.Frankfurt1.Level3.net (4.68.23.126) 8.490 ms 6 ae-41-41.ebr2.Washington1.Level3.net (4.69.137.50) 98.025 ms ae-82-82.ebr2.Frankfurt1.Level3.net (4.69.140.25) 8.644 ms ae-72-72.ebr2.Frankfurt1.Level3.net (4.69.140.21) 8.486 ms 7 ae-92-92.csw4.Washington1.Level3.net (4.69.134.158) 99.121 ms ae-43-43.ebr2.Washington1.Level3.net (4.69.137.58) 98.032 ms ae-62-62.csw1.Washington1.Level3.net (4.69.134.146) 105.569 ms 8 ae-82-82.csw3.Washington1.Level3.net (4.69.134.154) 104.113 ms * ae-72-72.csw2.Washington1.Level3.net (4.69.134.150) 99.957 ms 9 * * ae-71-71.ebr1.Washington1.Level3.net (4.69.134.133) 111.362 ms 10 * * ae-7.ebr3.Dallas1.Level3.net (4.69.134.21) 138.031 ms 11 ae-7.ebr3.Dallas1.Level3.net (4.69.134.21) 133.965 ms ae-93-93.csw4.Dallas1.Level3.net (4.69.136.166) 135.473 ms * 12 ae-93-93.csw4.Dallas1.Level3.net (4.69.136.166) 134.879 ms ae-12-69.car2.Dallas1.Level3.net (4.68.19.4) 132.669 ms ae-63-63.csw1.Dallas1.Level3.net (4.69.136.154) 141.555 ms 13 DATABANK-HO.car2.Dallas1.Level3.net (4.71.170.2) 132.499 ms ae-12-69.car2.Dallas1.Level3.net (4.68.19.4) 132.503 ms DATABANK-HO.car2.Dallas1.Level3.net (4.71.170.2) 133.758 ms 14 DATABANK-HO.car2.Dallas1.Level3.net (4.71.170.2) 133.432 ms 63.164.96.62 (63.164.96.62) 132.821 ms 63.164.96.54 (63.164.96.54) 133.286 ms 15 63.164.96.54 (63.164.96.54) 133.754 ms 63.164.96.242 (63.164.96.242) 132.972 ms 63.164.96.54 (63.164.96.54) 136.101 ms 16 * 63.164.96.242 (63.164.96.242) 133.342 ms * 17 * * 162.194.232.72.static.reverse.ltdomains.com (72.232.194.162) 133.028 ms !Z 18 162.194.232.72.static.reverse.ltdomains.com (72.232.194.162) 133.135 ms !Z 132.824 ms !Z 134.067 ms !Z
This got me thinking and I ran mtr (with -c 10 -r):
srv3# mtr -c 10 -r centos.org HOST: srv3.ultra-secure.de Loss% Snt Last Avg Best Wrst StDev 1. gw.rtr2.colo1.NBG1.content-c 0.0% 10 0.3 0.4 0.3 0.5 0.0 2. p10-92.cr2.NBG1.content-core 0.0% 10 0.3 0.3 0.3 0.3 0.0 3. te-7-2.edge3.frf1.Level3.net 0.0% 10 8.3 8.3 8.3 8.3 0.0 4. vlan99.csw4.Frankfurt1.Level 0.0% 10 8.6 12.6 8.4 20.6 5.0 5. ae-92-92.ebr2.Frankfurt1.Lev 0.0% 10 8.4 8.7 8.4 9.5 0.3 6. ae-42-42.ebr2.Washington1.Le 0.0% 10 96.8 96.8 96.7 97.0 0.1 7. ae-92-92.csw4.Washington1.Le 0.0% 10 98.6 102.3 97.0 110.3 5.0 8. ae-91-91.ebr1.Washington1.Le 50.0% 10 101.8 103.6 98.6 109.8 4.1 9. ae-2.ebr3.Atlanta2.Level3.ne 30.0% 10 115.2 118.9 114.1 124.7 4.4 10. ae-7.ebr3.Dallas1.Level3.net 20.0% 10 132.4 135.5 130.9 142.2 4.7 11. ae-73-73.csw2.Dallas1.Level3 0.0% 10 143.9 137.7 131.2 143.9 4.9 12. ae-22-79.car2.Dallas1.Level3 0.0% 10 131.0 196.7 131.0 318.9 77.2 13. DATABANK-HO.car2.Dallas1.Lev 0.0% 10 131.0 131.2 130.8 131.7 0.3 14. 63.164.96.62 0.0% 10 131.5 131.3 130.8 131.5 0.2 15. 63.164.96.162 10.0% 10 131.9 131.8 131.4 132.4 0.3 16. ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 17. 162.194.232.72.static.revers 10.0% 10 131.2 131.5 131.2 131.7 0.1 srv3#
So, from this point of view, L3 seems to be losing some packets.
Regards, Rainer
From Sweden it stops at layertech.com:
Viamac:~ per$ traceroute centos.org traceroute to centos.org (72.232.194.162), 64 hops max, 40 byte packets 1 10.99.99.1 (10.99.99.1) 1.257 ms 0.352 ms 0.356 ms 2 192.168.150.1 (192.168.150.1) 1.019 ms 1.159 ms 1.216 ms 3 212.247.10.238 (212.247.10.238) 1.716 ms 1.579 ms 1.447 ms 4 htg40.pos2-0s54point-to-point.swip.net (130.244.9.9) 3.054 ms 3.224 ms 3.211 ms 5 htg0-ncore-1.gigabiteth1-24.swip.net (130.244.189.1) 4.972 ms 3.261 ms 3.183 ms 6 htg0-core-1.gigabiteth1-0-0.swip.net (130.244.52.129) 3.276 ms 3.118 ms 3.144 ms 7 kst-core-1.gigabiteth8-0-0.swip.net (130.244.218.154) 3.536 ms 3.205 ms 3.399 ms 8 gbg-core-1.pos8-0-0.swip.net (130.244.39.142) 9.549 ms 9.606 ms 9.633 ms 9 ams17-core-1.pos6-0-0.swip.net (130.244.205.150) 24.411 ms 24.287 ms 24.195 ms 10 nyc9-core-1.pos6-0-0.swip.net (130.244.218.142) 101.672 ms 101.449 ms 101.265 ms 11 sl-gw35-nyc-2-0-1.sprintlink.net (144.223.156.197) 101.756 ms 101.771 ms 101.440 ms 12 sl-crs2-nyc-0-10-2-0.sprintlink.net (144.232.13.38) 102.922 ms 102.690 ms 102.697 ms 13 sl-crs2-chi-0-10-2-0.sprintlink.net (144.232.20.118) 125.609 ms 124.491 ms 124.373 ms 14 sl-crs1-kc-0-13-5-0.sprintlink.net (144.232.20.231) 135.012 ms 134.917 ms 134.657 ms 15 144.232.20.56 (144.232.20.56) 146.881 ms 146.626 ms 146.111 ms 16 sl-st30-dal-0-4-0-0.sprintlink.net (144.232.19.176) 147.995 ms 147.763 ms 147.911 ms 17 sl-lodst3-231662-0.sprintlink.net (144.223.244.194) 147.746 ms 147.603 ms 147.544 ms 18 ae_cw_10g.databank.com (63.164.96.62) 147.226 ms 146.911 ms aw_cw_10g.databank.com (63.164.96.54) 147.865 ms 19 pod22d_aw.layeredtech.com (63.164.96.206) 147.486 ms pod22g_aw.layeredtech.com (63.164.96.158) 148.170 ms pod22d_aw.layeredtech.com (63.164.96.206) 147.356 ms 20 * * * 21 * * * 22 * * * 23 * * *
On 3/20/09 4:16 PM, "Rainer Duffner" rainer@ultra-secure.de wrote:
mtr -c 10 -r centos.org
On Fri, Mar 20, 2009 at 10:16 AM, Rainer Duffner rainer@ultra-secure.de wrote:
Lanny Marcus schrieb:
On Mon, Mar 16, 2009 at 12:17 PM, Lanny Marcus lmmailinglists@gmail.com wrote:
The web site for CentOS is not responding. I can ping it OK, but the web site is not responding to http requests from my browser. If the webmaster should happen to read this...
As I posted late yesterday afternoon, the problem went away a few days ago, but then began again, yesterday. I can ping the server and traceroute to it, but no http for me from centos.org again.
<snip>
FYI: this is what I get from my colo'ed box in >Germany:
<snip>
This got me thinking and I ran mtr (with -c 10 -r):
Rainer: Thank you! mtr is a new command for me. Here's what I got;
[root@dell2400 ~]# mtr -c 10 -r centos.org dell2400.homelan Snt: 10 Loss% Last Avg Best Wrst StDev ipcop233 0.0% 0.3 0.4 0.3 1.5 0.4 192.168.1.1 0.0% 1.0 1.1 0.8 3.2 0.7 dsl-emcali-190.1.240.1.emcali.net.co 0.0% 12.8 15.7 12.8 17.4 1.5 172.16.1.3 0.0% 13.5 21.5 12.0 57.1 14.7 190.90.2.25 0.0% 36.4 74.4 36.4 173.0 46.3 so-4-2-3-nmi-core01.nwnnetwork.net 0.0% 73.9 81.0 73.9 124.4 15.4 ge-1-1-0-nmi-core02.columbus-networks.com 0.0% 74.5 79.7 74.5 85.7 3.9 xe-8-0-0.edge2.Miami1.Level3.net 0.0% 81.3 84.7 81.3 106.1 7.5 ae-31-51.ebr1.Miami1.Level3.net 0.0% 87.5 89.9 84.0 105.2 6.4 ae-2.ebr1.Dallas1.Level3.net 0.0% 117.2 119.7 110.0 140.1 10.2 ae-61-61.csw1.Dallas1.Level3.net 0.0% 111.8 116.1 110.7 125.6 4.8 ae-12-69.car2.Dallas1.Level3.net 0.0% 111.6 145.3 109.9 249.0 55.1 DATABANK-HO.car2.Dallas1.Level3.net 0.0% 112.0 112.7 109.7 119.4 2.9 ae_cw_10g.databank.com 0.0% 111.6 113.6 110.7 126.5 4.8 pod22h_ae.layeredtech.com 0.0% 116.8 115.5 110.7 139.7 8.7 ??? 100.0 0.0 0.0 0.0 0.0 0.0 162.194.232.72.static.reverse.ltdomains.com 10.0% 111.4 117.4 110.9 140.3 9.8 [root@dell2400 ~]#
So, from this point of view, L3 seems to be losing >some packets.
At the end, when I ran the mtr command you used, ??? is losing 100% at one router within layered tech.com (the hop after pod22h_ae.layeredtech.com) and the next one, 162.194.232.72.static.reverse.ltdomains.com is losing10%?
In the loss column, the 2 at the end are within the Layered Tech DC, so it does not look like this is a problem with Level3.
Thank you!
On Fri, Mar 20, 2009 at 11:02:46AM -0500, Lanny Marcus wrote:
ae_cw_10g.databank.com 0.0% 111.6 113.6 110.7 126.5 4.8 pod22h_ae.layeredtech.com 0.0% 116.8 115.5 110.7 139.7 8.7 ??? 100.0 0.0 0.0 0.0 0.0 0.0 162.194.232.72.static.reverse.ltdomains.com 10.0% 111.4 117.4 110.9 140.3 9.8 [root@dell2400 ~]#
So, from this point of view, L3 seems to be losing >some packets.
At the end, when I ran the mtr command you used, ??? is losing 100% at one router within layered tech.com (the hop after pod22h_ae.layeredtech.com) and the next one, 162.194.232.72.static.reverse.ltdomains.com is losing10%?
162.194.232.72.static.reverse.ltdomains.com is www.centos.org
Tru
2009/3/20 Tru Huynh tru@centos.org:
On Fri, Mar 20, 2009 at 11:02:46AM -0500, Lanny Marcus wrote:
ae_cw_10g.databank.com 0.0% 111.6 113.6 110.7 126.5 4.8 pod22h_ae.layeredtech.com 0.0% 116.8 115.5 110.7 139.7 8.7 ??? 100.0 0.0 0.0 0.0 0.0 0.0 162.194.232.72.static.reverse.ltdomains.com 10.0% 111.4 117.4 110.9 140.3 9.8 [root@dell2400 ~]#
So, from this point of view, L3 seems to be losing >some packets.
At the end, when I ran the mtr command you used, ??? is losing 100% at one router within layered tech.com (the hop after pod22h_ae.layeredtech.com) and the next one, 162.194.232.72.static.reverse.ltdomains.com is losing10%?
162.194.232.72.static.reverse.ltdomains.com is www.centos.org
Tru: And it's losing 10%? There seems to be a larger problem, before it gets to that server, within LT. If you read the last few messages in this thread, posted this morning (GMT -5 here), by me, Rainer and Per, the LT Tech should be able to track down the problem(s) within their DC. Take care and thank you! Lanny
On 3/20/09, Lanny Marcus lmmailinglists@gmail.com wrote:
2009/3/20 Tru Huynh tru@centos.org:
On Fri, Mar 20, 2009 at 11:02:46AM -0500, Lanny Marcus wrote:
ae_cw_10g.databank.com 0.0% 111.6 113.6 110.7 126.5 4.8 pod22h_ae.layeredtech.com 0.0% 116.8 115.5 110.7 139.7 8.7 ??? 100.0 0.0 0.0 0.0 0.0 0.0 162.194.232.72.static.reverse.ltdomains.com 10.0% 111.4 117.4 110.9 140.3 9.8 [root@dell2400 ~]#
So, from this point of view, L3 seems to be losing >some packets.
At the end, when I ran the mtr command you used, ??? is losing 100% at one router within layered tech.com (the hop after pod22h_ae.layeredtech.com) and the next one, 162.194.232.72.static.reverse.ltdomains.com is losing10%?
162.194.232.72.static.reverse.ltdomains.com is www.centos.org
Tru: And it's losing 10%? There seems to be a larger problem, before it gets to that server, within LT. If you read the last few messages in this thread, posted this morning (GMT -5 here), by me, Rainer and Per, the LT Tech should be able to track down the problem(s) within their DC. Take care and thank you! Lanny
Tru: Until Rainer showed the command he used "mtr -c 10 -r centos.org" I was unaware of it, until this morning. Possibly the 10% loss at the server is OK? I just used that command, to both of my web sites in CT. They are on shared hosting, in different DCs and with different network carriers. Both of them have that 10% loss shown. The traceroute to centos.org that Per did from Sweden this morning seemed to die, within LayeredTech, as I recall. I don't believe the problem to the CentOS server is on Level3, but I do see problems to my site which uses Level3, on 2 Routers in Atlanta and 1 in Washington.... Lanny
Lanny Marcus schrieb:
On 3/20/09, Lanny Marcus lmmailinglists@gmail.com wrote:
2009/3/20 Tru Huynh tru@centos.org:
On Fri, Mar 20, 2009 at 11:02:46AM -0500, Lanny Marcus wrote:
ae_cw_10g.databank.com 0.0% 111.6 113.6 110.7 126.5 4.8 pod22h_ae.layeredtech.com 0.0% 116.8 115.5 110.7 139.7 8.7 ??? 100.0 0.0 0.0 0.0 0.0 0.0 162.194.232.72.static.reverse.ltdomains.com 10.0% 111.4 117.4 110.9 140.3 9.8 [root@dell2400 ~]#
So, from this point of view, L3 seems to be losing >some packets.
At the end, when I ran the mtr command you used, ??? is losing 100% at one router within layered tech.com (the hop after pod22h_ae.layeredtech.com) and the next one, 162.194.232.72.static.reverse.ltdomains.com is losing10%?
162.194.232.72.static.reverse.ltdomains.com is www.centos.org
Tru: And it's losing 10%? There seems to be a larger problem, before it gets to that server, within LT. If you read the last few messages in this thread, posted this morning (GMT -5 here), by me, Rainer and Per, the LT Tech should be able to track down the problem(s) within their DC. Take care and thank you! Lanny
Tru: Until Rainer showed the command he used "mtr -c 10 -r centos.org" I was unaware of it, until this morning. Possibly the 10% loss at the server is OK?
This can have a variety of reasons - packet-filtering is not the least likely ;-) I don't think centos.org is just sitting there with no iptables etc.
I can reach centos.org from my colo-box perfectly (via lynx) - but I'm not sure why I would need 16 or 17 hops to do that - and even 20 for layeredtech.com. I can reach our own little website via 8 hops, same as yahoo.de
I should point out that I have no deeper insight into the inner workings of how internet-routing works - maybe this is all expected. But in the past, mtr was quite reliable pin-pointing severe problems.
Rainer
Rainer Duffner wrote:
srv3# mtr -c 10 -r centos.org HOST: srv3.ultra-secure.de Loss% Snt Last Avg Best Wrst StDev ... 7. ae-92-92.csw4.Washington1.Le 0.0% 10 98.6 102.3 97.0 110.3 5.0 8. ae-91-91.ebr1.Washington1.Le 50.0% 10 101.8 103.6 98.6 109.8 4.1 9. ae-2.ebr3.Atlanta2.Level3.ne 30.0% 10 115.2 118.9 114.1 124.7 4.4 10. ae-7.ebr3.Dallas1.Level3.net 20.0% 10 132.4 135.5 130.9 142.2 4.7 11. ae-73-73.csw2.Dallas1.Level3 0.0% 10 143.9 137.7 131.2 143.9 4.9 ... So, from this point of view, L3 seems to be losing some packets.
'lost packets' on traces from intermediate hops aren't significant, as most routers treat PING as a low priority and if they are busy doing their primary routing job, they'll ignore some percentage of pings.
traceroute and its ilk are very inexact tools. your MTR command, run from a San Jose, California machine, gives...
# mtr -c 10 -r centos.org XXXXXXXXX.com Snt: 10 Loss% Last Avg Best Wrst StDev ge-0-0-1-414.er2.sjc1.got.net 0.0% 2.5 22.0 2.5 191.4 59.5 ge-1-0-10.cr1.sjc1.got.net 0.0% 22.2 22.2 14.5 30.0 5.6 173.ge-1-3-0.mpr1.sjc2.us.above.net 0.0% 3.9 12.0 3.0 48.5 17.0 xe-1-1-0.er1.sjc7.us.above.net 0.0% 5.5 4.9 3.4 7.0 1.2 above-level3.sjc7.us.above.net 0.0% 66.3 11.1 3.2 66.3 19.5 vlan69.csw1.SanJose1.Level3.net 0.0% 7.6 9.7 4.5 17.0 4.2 ae-63-63.ebr3.SanJose1.Level3.net 0.0% 17.8 13.8 6.5 30.4 7.6 ae-2.ebr3.LosAngeles1.Level3.net 0.0% 23.6 18.1 12.5 25.4 5.2 ae-83-83.csw3.LosAngeles1.Level3.net 0.0% 24.4 20.3 13.3 25.3 4.9 ae-82-82.ebr2.LosAngeles1.Level3.net 10.0% 24.8 21.8 14.9 26.2 4.1 ae-3.ebr3.Dallas1.Level3.net 20.0% 47.6 48.8 46.7 56.1 3.0 ae-73-73.csw2.Dallas1.Level3.net 0.0% 46.1 50.3 46.1 58.4 4.3 ae-22-79.car2.Dallas1.Level3.net 0.0% 45.9 47.0 45.9 48.0 0.8 DATABANK-HO.car2.Dallas1.Level3.net 0.0% 47.6 47.4 46.2 49.0 1.0 ae_cw_10g.databank.com 0.0% 48.5 47.1 46.7 48.5 0.6 pod22c_ae.layeredtech.com 0.0% 46.9 47.5 46.5 48.2 0.5 ??? 100.0 0.0 0.0 0.0 0.0 0.0 162.194.232.72.static.reverse.ltdomains.com 0.0% 48.0 47.3 46.4 48.0 0.4
On 3/20/09, John R Pierce pierce@hogranch.com wrote:
Rainer Duffner wrote:
srv3# mtr -c 10 -r centos.org HOST: srv3.ultra-secure.de Loss% Snt Last Avg Best Wrst
<snip>
'lost packets' on traces from intermediate hops aren't significant, as most routers treat PING as a low priority and if they are busy doing their primary routing job, they'll ignore some percentage of pings.
traceroute and its ilk are very inexact tools. your MTR command, run from a San Jose, California machine, gives...
<snip> John: Then I believe the most important data we have are what you found a few nights ago, and the traceroute Per did from Sweden this morning (USA time) that died within the LayeredTech DC. Lanny
On Fri, 2009-03-20 at 13:08 -0500, Lanny Marcus wrote:
<snip> John: Then I believe the most important data we have are what you found a few nights ago, and the traceroute Per did from Sweden this morning (USA time) that died within the LayeredTech DC. Lanny
Adding to the data, from here in North Carolina
centos501.homegroannetwork Snt: 10 Loss% Last Avg Best Wrst StDev Firewall.HomeGroanNetwork 0.0% 0.1 0.5 0.1 2.8 0.8 10.96.64.1 0.0% 8.4 8.2 5.6 14.9 2.6 gig2-1-1.gnboncsg-rtr2.triad.rr.com 0.0% 6.9 7.8 6.2 9.3 1.2 ge-1-3-0.chrlncpop-rtr1.southeast.rr.com 0.0% 13.7 11.3 9.0 13.7 1.5 te-3-1.car1.Charlotte1.Level3.net 0.0% 12.2 11.6 9.5 14.5 1.5 ae-4-4.ebr1.Atlanta2.Level3.net 0.0% 19.9 20.7 14.4 28.0 4.9 ae-63-60.ebr3.Atlanta2.Level3.net 60.0% 25.0 23.2 18.9 25.0 2.9 ae-7.ebr3.Dallas1.Level3.net 40.0% 46.8 44.0 38.3 49.0 4.2 ae-73-73.csw2.Dallas1.Level3.net 0.0% 34.6 41.5 34.6 47.7 5.1 ae-22-79.car2.Dallas1.Level3.net 0.0% 38.0 36.6 34.7 38.1 1.2 DATABANK-HO.car2.Dallas1.Level3.net 0.0% 36.3 35.4 33.3 37.7 1.3 aw_cw_10g.databank.com 0.0% 35.0 35.7 33.3 37.7 1.5 pod22e_aw.layeredtech.com 0.0% 37.5 37.1 35.2 39.3 1.3 ??? 100.0 0.0 0.0 0.0 0.0 0.0 162.194.232.72.static.reverse.ltdomains.com 0.0% 36.2 36.8 35.3 38.5 1.1
<snip sig stuff>
HTH
William L. Maltby wrote:
On Fri, 2009-03-20 at 13:08 -0500, Lanny Marcus wrote:
<snip> John: Then I believe the most important data we have are what you found a few nights ago, and the traceroute Per did from Sweden this morning (USA time) that died within the LayeredTech DC. Lanny
Adding to the data, from here in North Carolina
centos501.homegroannetwork Snt: 10 Loss% Last Avg Best Wrst StDev Firewall.HomeGroanNetwork 0.0% 0.1 0.5 0.1 2.8 0.8 10.96.64.1 0.0% 8.4 8.2 5.6 14.9 2.6 gig2-1-1.gnboncsg-rtr2.triad.rr.com 0.0% 6.9 7.8 6.2 9.3 1.2 ge-1-3-0.chrlncpop-rtr1.southeast.rr.com 0.0% 13.7 11.3 9.0 13.7 1.5 te-3-1.car1.Charlotte1.Level3.net 0.0% 12.2 11.6 9.5 14.5 1.5 ae-4-4.ebr1.Atlanta2.Level3.net 0.0% 19.9 20.7 14.4 28.0 4.9 ae-63-60.ebr3.Atlanta2.Level3.net 60.0% 25.0 23.2 18.9 25.0 2.9 ae-7.ebr3.Dallas1.Level3.net 40.0% 46.8 44.0 38.3 49.0 4.2 ae-73-73.csw2.Dallas1.Level3.net 0.0% 34.6 41.5 34.6 47.7 5.1 ae-22-79.car2.Dallas1.Level3.net 0.0% 38.0 36.6 34.7 38.1 1.2 DATABANK-HO.car2.Dallas1.Level3.net 0.0% 36.3 35.4 33.3 37.7 1.3 aw_cw_10g.databank.com 0.0% 35.0 35.7 33.3 37.7 1.5 pod22e_aw.layeredtech.com 0.0% 37.5 37.1 35.2 39.3 1.3 ??? 100.0 0.0 0.0 0.0 0.0 0.0 162.194.232.72.static.reverse.ltdomains.com 0.0% 36.2 36.8 35.3 38.5 1.1
<snip sig stuff>
Another data point. I'm on sbcglobal DSL near Dallas. (rcsntx is Richardson Texas, an adjacent suburb of Dallas.) Without further comment:
[root@mavis rj]# mtr -c 10 -r centos.org mavis.localdomain Snt: 10 Loss% Last Avg Best Wrst StDev merguetroid.localdomain 0.0% 0.7 0.8 0.6 1.5 0.3 bras3-l0.rcsntx.sbcglobal.net 0.0% 13.7 14.1 13.4 16.7 1.0 dist2-vlan130.rcsntx.sbcglobal.net 0.0% 12.0 11.8 11.4 12.3 0.3 bb2.10g3-0.rcsntx.sbcglobal.net 10.0% 11.6 18.9 11.6 60.6 16.4 ex1-p2-0.eqdltx.sbcglobal.net 0.0% 13.2 13.3 11.8 16.8 1.7 asn2828-xo.pxnyny.sbcglobal.net 0.0% 61.9 19.0 12.0 61.9 15.9 ae-42-99.car2.Dallas1.Level3.net 0.0% 14.8 19.9 12.4 76.6 20.0 DATABANK-HO.car2.Dallas1.Level3.net 0.0% 12.7 12.9 12.3 13.4 0.3 aw_cw_10g.databank.com 10.0% 13.2 13.0 12.7 13.3 0.2 pod22e_aw.layeredtech.com 0.0% 12.9 13.5 12.5 16.2 1.1 ??? 100.0 0.0 0.0 0.0 0.0 0.0 162.194.232.72.static.reverse.ltdomains.com 0.0% 13.9 13.1 12.6 13.9 0.4
On Fri, Mar 20, 2009 at 01:58:13PM -0500, Robert wrote:
Another data point. I'm on sbcglobal DSL near Dallas. (rcsntx is Richardson Texas, an adjacent suburb of Dallas.) Without further comment:
[root@mavis rj]# mtr -c 10 -r centos.org mavis.localdomain Snt: 10 Loss% Last Avg Best Wrst StDev merguetroid.localdomain 0.0% 0.7 0.8 0.6 1.5 0.3 bras3-l0.rcsntx.sbcglobal.net 0.0% 13.7 14.1 13.4 16.7 1.0 dist2-vlan130.rcsntx.sbcglobal.net 0.0% 12.0 11.8 11.4 12.3 0.3 bb2.10g3-0.rcsntx.sbcglobal.net 10.0% 11.6 18.9 11.6 60.6 16.4 ex1-p2-0.eqdltx.sbcglobal.net 0.0% 13.2 13.3 11.8 16.8 1.7 asn2828-xo.pxnyny.sbcglobal.net 0.0% 61.9 19.0 12.0 61.9 15.9 ae-42-99.car2.Dallas1.Level3.net 0.0% 14.8 19.9 12.4 76.6 20.0 DATABANK-HO.car2.Dallas1.Level3.net 0.0% 12.7 12.9 12.3 13.4 0.3 aw_cw_10g.databank.com 10.0% 13.2 13.0 12.7 13.3 0.2 pod22e_aw.layeredtech.com 0.0% 12.9 13.5 12.5 16.2 1.1 ??? 100.0 0.0 0.0 0.0 0.0 0.0 162.194.232.72.static.reverse.ltdomains.com 0.0% 13.9 13.1 12.6 13.9 0.4
mtr from Philadelphia, PA, Level-3 outbound:
radagast.gerdesas.com Snt: 10 Loss% Last Avg Best Wrst StDev . 0.0% 0.1 0.1 0.1 0.1 0.0 cr2-cr1-g1-0-0-22-esw1.razorservers.net 0.0% 0.4 0.4 0.4 0.5 0.1 ge-7-5.car1.Philadelphia1.Level3.net 0.0% 0.4 57.9 0.4 197.5 67.3 ae-7-7.ebr2.Washington1.Level3.net 0.0% 3.9 3.7 3.6 3.9 0.1 ae-62-62.csw1.Washington1.Level3.net 0.0% 8.0 8.8 3.7 13.9 4.1 ae-61-61.ebr1.Washington1.Level3.net 40.0% 11.1 9.9 4.2 16.3 4.8 ae-2.ebr3.Atlanta2.Level3.net 50.0% 17.9 20.8 17.9 24.3 3.0 ae-7.ebr3.Dallas1.Level3.net 60.0% 37.8 43.9 37.8 51.0 6.2 ae-83-83.csw3.Dallas1.Level3.net 0.0% 48.9 44.0 38.0 50.4 5.1 ae-32-89.car2.Dallas1.Level3.net 40.0% 38.0 97.1 38.0 222.9 75.7 DATABANK-HO.car2.Dallas1.Level3.net 0.0% 38.4 38.3 38.1 38.5 0.1 ae_cw_10g.databank.com 0.0% 38.4 38.5 38.1 38.9 0.2 pod22c_ae.layeredtech.com 0.0% 38.6 38.8 38.5 39.4 0.3 ??? 100.0 0.0 0.0 0.0 0.0 0.0 162.194.232.72.static.reverse.ltdomains.com 0.0% 38.6 38.5 38.3 38.7 0.1
mtr from Chicago, IL, NTT outbound:
HOST LOSS RCVD SENT BEST AVG WORST wrt1.beanproducts.net 0% 10 10 0.57 0.72 1.10 72.54.40.205 0% 10 10 1.70 1.94 2.59 172.21.38.25 0% 10 10 5.87 6.06 6.19 192.168.42.2 0% 10 10 6.11 6.62 7.60 128.242.180.5 0% 10 10 6.35 118.34 204.39 xe-2-1-0.r20.chcgil09.us.bb.gin.ntt.net 0% 10 10 6.27 6.52 7.34 sl-st21-chi-14-1-1.sprintlink.net 0% 10 10 6.39 6.61 7.54 sl-crs2-chi-0-1-0-0.sprintlink.net 0% 10 10 7.28 7.82 9.31 sl-crs1-kc-0-14-3-0.sprintlink.net 0% 10 10 17.97 18.91 19.54 144.232.20.56 0% 10 10 30.96 31.64 32.18 sl-st30-dal-0-4-0-0.sprintlink.net 0% 10 10 30.73 31.27 31.91 sl-lodst3-231662-0.sprintlink.net 0% 10 10 30.50 30.89 32.01 ae_cw_10g.databank.com 0% 10 10 30.64 30.88 31.88 pod22f_ae.layeredtech.com 0% 10 10 30.59 32.21 41.55 ??? 100% 0 10 0.00 0.00 0.00 162.194.232.72.static.reverse.ltdomains.com 0% 10 10 30.75 31.11 31.43
mtr from Manchester, TN, Charter outbound:
tap.gerdesas.com Snt: 10 Loss% Last Avg Best Wrst StDev ??? 100.0 0.0 0.0 0.0 0.0 0.0 static.unknown.charter.com 0.0% 8.5 15.1 6.0 36.9 11.3 static.unknown.charter.com 0.0% 20.6 22.0 7.6 69.3 24.0 static.unknown.charter.com 0.0% 11.4 24.5 10.8 98.7 27.6 24-159-64-1.static.jcsn.tn.charter.com 0.0% 12.5 80.1 11.1 183.8 73.1 static.unknown.charter.com 0.0% 11.4 20.2 11.2 53.4 13.6 atx-edge-02.inet.qwest.net 0.0% 31.4 73.1 30.4 364.1 103.8 atl-core-01.inet.qwest.net 0.0% 36.3 50.7 30.1 181.4 46.9 atl-brdr-04.inet.qwest.net 0.0% 83.3 54.7 29.9 156.2 41.3 sl-crs2-atl-0-1-5-1.sprintlink.net 0.0% 49.6 59.0 31.8 155.8 40.9 sl-crs2-fw-0-12-0-2.sprintlink.net 0.0% 56.0 77.7 54.9 126.6 23.8 sl-st30-dal-0-12-0-0.sprintlink.net 0.0% 90.9 78.2 55.0 120.0 21.3 sl-lodst3-231662-0.sprintlink.net 0.0% 78.7 74.8 56.0 151.4 29.0 ae_cw_10g.databank.com 0.0% 123.1 74.9 54.4 123.1 24.1 pod22f_ae.layeredtech.com 0.0% 92.4 80.2 53.5 117.8 22.8 ??? 100.0 0.0 0.0 0.0 0.0 0.0 162.194.232.72.static.reverse.ltdomains.com 0.0% 54.1 69.6 53.4 136.7 29.1
mtr from Boston, MA, Level-3 outbound:
HOST: neural Loss% Snt Last Avg Best Wrst StDev 1. 2n+1-m160-1.optr.net 0.0% 10 0.3 0.4 0.3 0.5 0.1 2. ge-7-20-245.car1.Boston1.Lev 0.0% 10 0.5 0.4 0.4 0.6 0.1 3. ae-2-7.bar1.Boston1.Level3.n 0.0% 10 0.3 9.8 0.3 95.2 30.0 4. ae-0-11.bar2.Boston1.Level3. 0.0% 10 0.3 0.7 0.3 4.4 1.3 5. ae-8-8.ebr1.NewYork1.Level3. 0.0% 10 5.0 5.0 5.0 5.1 0.0 6. ae-91-91.csw4.NewYork1.Level 0.0% 10 13.0 9.7 5.3 17.4 4.3 7. ae-94-94.ebr4.NewYork1.Level 0.0% 10 10.3 10.8 5.8 18.7 4.7 8. ae-3.ebr2.Dallas1.Level3.net 0.0% 10 40.1 44.0 39.9 52.9 4.7 9. ae-62-62.csw1.Dallas1.Level3 0.0% 10 40.2 46.2 40.2 53.3 4.5 10. ae-12-69.car2.Dallas1.Level3 0.0% 10 40.4 42.9 40.3 64.4 7.5 11. DATABANK-HO.car2.Dallas1.Lev 0.0% 10 40.9 40.6 40.3 40.9 0.2 12. aw_cw_10g.databank.com 0.0% 10 41.0 40.6 40.3 41.0 0.2 13. pod22a_aw.layeredtech.com 0.0% 10 41.0 41.1 40.6 43.0 0.7 14. ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 15. 162.194.232.72.static.revers 0.0% 10 41.0 40.7 40.4 41.0 0.2
John
John R. Dennison wrote:
162.194.232.72.static.reverse.ltdomains.com 0.0% 13.9 13.1 12.6 13.9 0.4
162.194.232.72.static.reverse.ltdomains.com 0.0% 38.6 38.5 38.3 38.7 0.1 162.194.232.72.static.reverse.ltdomains.com 0% 10 10 30.75 31.11 31.43 162.194.232.72.static.reverse.ltdomains.com 0.0% 54.1 69.6 53.4 136.7 29.1 15. 162.194.232.72.static.revers 0.0% 10 41.0 40.7 40.4 41.0 0.2
And all those plus the ones in the mails you and people before you sent do reach www.centos.org. What are you trying to prove here? Why is nobody talking to layered tech directly, if he/she cannot reach www.centos.org?
I really do not understand this thread anymore.
Ralph
On 3/20/09, Ralph Angenendt ra+centos@br-online.de wrote:
John R. Dennison wrote:
162.194.232.72.static.reverse.ltdomains.com 0.0% 13.9 13.1 12.6
<snip.
And all those plus the ones in the mails you and people before you sent do reach www.centos.org. What are you trying to prove here? Why is nobody talking to layered tech directly, if he/she cannot reach www.centos.org?
I really do not understand this thread anymore.
Ralph: Tru opened a Trouble Ticket, at Layered Tech, several days ago. The LT Support Tech replied he/she thought there was a problem on Level3 in Dallas. It began working for me shortly after the Trouble Ticket was filed (I thought they had fixed something.....), but then died, a day or so later, again. Tru reopened the Trouble Ticket and the LT Tech replied that they could not duplicate the problem (I understand how hard it is to troubleshoot intermittent problems), they had not fixed anything and they felt that it was a problem on Level 3, in Dallas. Probably this thread should die, until more people cannot surf the site. That's what I'm getting, although I can ping it and traceroute to it. As I said, it's an intermittent problem. Lanny
Lanny Marcus wrote:
On 3/20/09, Ralph Angenendt ra+centos@br-online.de wrote:
Probably this thread should die, until more people cannot surf the site. That's what I'm getting, although I can ping it and traceroute to it. As I said, it's an intermittent problem. Lanny
I haven't even seen one try at showing a tcpdump or similar.
Ralph
On 3/20/09, Ralph Angenendt ra+centos@br-online.de wrote:
Lanny Marcus wrote:
On 3/20/09, Ralph Angenendt ra+centos@br-online.de wrote: Probably this thread should die, until more people cannot surf the site. That's what I'm getting, although I can ping it and traceroute to it. As I said, it's an intermittent problem. Lanny
I haven't even seen one try at showing a tcpdump or similar.
If you will give me the commands you want me to try, I'll do it. If I get a lot of output from the commands you give me, I put it on pastebin or something like that?
On Mar 20, 2009, at 6:35 PM, Lanny Marcus lmmailinglists@gmail.com wrote:
On 3/20/09, Ralph Angenendt ra+centos@br-online.de wrote:
Lanny Marcus wrote:
On 3/20/09, Ralph Angenendt ra+centos@br-online.de wrote: Probably this thread should die, until more people cannot surf the site. That's what I'm getting, although I can ping it and traceroute to it. As I said, it's an intermittent problem. Lanny
I haven't even seen one try at showing a tcpdump or similar.
If you will give me the commands you want me to try, I'll do it. If I get a lot of output from the commands you give me, I put it on pastebin or something like that?
It may be a blackhole router between you and the website. Some path that has a slightly smaller MTU but has ICMP disabled so the need to fragment messages aren't sent.
Try pinging with a 1500 byte packet and see if it reaches. If it doesn't decrease the size until it does and you then know what the short MTU size is and you can set that as your MTU on your outside interface of your router. A "workaround" solution.
-Ross
On Fri, Mar 20, 2009 at 10:05 PM, Ross Walker rswwalker@gmail.com wrote:
On Mar 20, 2009, at 6:35 PM, Lanny Marcus lmmailinglists@gmail.com
Probably this thread should die, until more people cannot surf the site. That's what I'm getting, although I can ping it and traceroute to it. As I said, it's an intermittent problem. Lanny
I haven't even seen one try at showing a tcpdump or similar.
<snip>
Ross: I have an old (2003) book about Network Troubleshooting. It shows an example of using Telnet to Port 80, to see what's happening. I just tried that, and assuming the CentOS server in Layered Tech is configured the same way, which is a huge assumption, it is not responding with HTML. However, I also tried that, to one of my web sites, on a shared server in CT, (I had them disable Telnet and Anonymous FTP) and it does not respond with HTML there either. My browser can load my web site without any problem. Probably this is *NOT* a valid way to test this! Here's what I got, in my 3 attempts:
(1) [lanny@dell2400 ~]$ telnet 72.232.194.162 80 Trying 72.232.194.162... Connected to 162.194.232.72.static.reverse.ltdomains.com (72.232.194.162). Escape character is '^]'. Connection closed by foreign host. [lanny@dell2400 ~]$
I did not type the GET command and it eventually timed out
(2) [lanny@dell2400 ~]$ telnet 72.232.194.162 80 Trying 72.232.194.162... Connected to 162.194.232.72.static.reverse.ltdomains.com (72.232.194.162). Escape character is '^]'. GET /
I got no response after that and no HTML content. This is the example in the book.
(3) [lanny@dell2400 ~]$ telnet 72.232.194.162 80 Trying 72.232.194.162... Connected to 162.194.232.72.static.reverse.ltdomains.com (72.232.194.162). Escape character is '^]'. GET
I got no response after that and no HTML content
It may be a blackhole router between you and the website. Some path that has a slightly smaller MTU but has ICMP disabled so the need to fragment messages aren't sent.
When I use the mtr -c 10 -r centos.org someone suggested yesterday, which John Pierce said might not be meaningful with this problem, within Layered Tech, I *always* see one hop in Dallas, with "???" and a 100% loss. What does that mean, if anything, with this problem?
ae_cw_10g.databank.com 0.0% 113.5 113.7 110.4 116.0 1.9 pod22h_ae.layeredtech.com 0.0% 115.2 114.3 112.5 115.4 1.0 ??? 100.0 0.0 0.0 0.0 0.0 0.0 162.194.232.72.static.reverse.ltdomains.com 10.0% 113.4 113.0 111.4 115.6 1.4 [root@dell2400 ~]#
Try pinging with a 1500 byte packet and see if it reaches. If it doesn't decrease the size until it does and you then know what the short MTU size is and you can set that as your MTU on your outside interface of your router. A "workaround" solution.
I used this command: ping -s 1500 centos.org and it does work OK. I get back 1508 bytes, which I believe is correct.
Please give me the exact command you want me to try, if that isn't it! I have been thinking about MTU, vaguely remembering things I did years ago, during the past few days.
The problem went away, for approximately 24 hours, after Tru filed the initial Trouble Ticket at Layered Tech, but it came back, a day or two later and now I know that Layered Tech didn't change anything and if it is on Level3 as the LT tech thought, they are probably unaware of a problem on their network. It's intermittent from my end. I was able to load the web pages from centos.org on Tuesday after the Trouble Ticket was filed at LT and on Wednesday as I recall and then yesterday, no joy again.
Your time and help, and that of everyone else in this thread, are much appreciated! Lanny
On Fri, Mar 20, 2009 at 10:49 PM, Lanny Marcus lmmailinglists@gmail.com wrote: <snip>
I haven't even seen one try at showing a tcpdump or similar.
<snip> Ross: I have an old (2003) book about Network Troubleshooting. It shows an example of using Telnet to Port 80, to see what's happening. I just tried that, and assuming the CentOS server in Layered Tech is configured the same way, which is a huge assumption, it is not responding with HTML. However, I also tried that, to one of my web sites, on a shared server in CT, (I had them disable Telnet and Anonymous FTP) and it does not respond with HTML there either. My browser can load my web site without any problem.
CORRECTION! After I replied to Ross, I went back to the terminal window and it DID work as the book shows, on my web site in CT. I see all the HTML. It must take a *long* time. I am going to try it again, on the centos.org web site
Lanny:
Ross: I have an old (2003) book about Network Troubleshooting. It shows an example of using Telnet to Port 80, to see what's happening. I just tried that, and assuming the CentOS server in Layered Tech is configured the same way, which is a huge assumption, it is not responding with HTML. However, I also tried that, to one of my web sites, on a shared server in CT, (I had them disable Telnet and Anonymous FTP) and it does not respond with HTML there either. My browser can load my web site without any problem. Probably this is *NOT* a valid way to test this! Here's what I got, in my 3 attempts:
insufficent info on your GET, and you can't use any backspaces, you pretty much have to paste a perfect command...
$ telnet www.centos.org www Trying 72.232.194.162... Connected to www.centos.org (72.232.194.162). Escape character is '^]'. HEAD / HTTP/1.1 Host: www.centos.org
HTTP/1.1 200 OK Date: Sat, 21 Mar 2009 04:08:16 GMT Server: Apache/2.0.52 (CentOS) X-Powered-By: PHP/4.3.9 Set-Cookie: PHPSESSID=d329988a9d984ee4bbe2e1e2f12d2aa0; path=/ Expires: Mon, 26 Jul 1997 05:00:00 GMT Cache-Control: private, no-cache Pragma: no-cache Connection: close Content-Type: text/html; charset=ISO-8859-1
Connection closed by foreign host.
note i had to send THREE lines... I used the HEAD command rather than GET so I don't have to stare at a bunch of html.
HEAD / HTTP/1.1 Host: www.centos.org
(and a blank line to terminate the request)
On Fri, Mar 20, 2009 at 11:12 PM, John R Pierce pierce@hogranch.com wrote: <snip> The telnets into my 2 web sites in CT with the GET did result in me seeing the HTML code, as the book shows.
insufficient info on your GET, and you can't use any backspaces, you pretty much have to paste a perfect command...
$ telnet www.centos.org www Trying 72.232.194.162... Connected to www.centos.org (72.232.194.162). Escape character is '^]'. HEAD / HTTP/1.1 Host: www.centos.org
<snip>
note i had to send THREE lines... I used the HEAD command rather than GET so I don't have to stare at a bunch of html.
HEAD / HTTP/1.1 Host: www.centos.org
(and a blank line to terminate the request)
OK John. I got that to work on my site in Trumbull, CT. :-) Also, on the one in Stratford, CT :-)
But, so far, I cannot get it to work with centos.org
From my box, the centos.org server is not responding.
[lanny@dell2400 ~]$ telnet www.centos.org www Trying 72.232.194.162... Connected to www.centos.org (72.232.194.162). Escape character is '^]'. HEAD / HTTP/1.1 Host: www.centos.org
Question: Do you have another command I can try from here? Thank you very much and good evening!
On Fri, 2009-03-20 at 23:57 -0500, Lanny Marcus wrote:
Question: Do you have another command I can try from here? Thank you very much and good evening!
Question for you Lanny. Do you have any ones computer you can use beside yours in the same place (town and country) that you live? If so try to access the sight with it. Make sure it's on the same Internet Network as yours, just to rule out your equipment. Do this before going any farther.
JohnStanley
On Sat, Mar 21, 2009 at 1:04 AM, JohnS jses27@gmail.com wrote:
On Fri, 2009-03-20 at 23:57 -0500, Lanny Marcus wrote:
<snip>
Question for you Lanny. Do you have any ones computer you can use beside yours in the same place (town and country) that you live? If so try to access the sight with it. Make sure it's on the same Internet Network as yours, just to rule out your equipment. Do this before going any farther.
John: After they leave today, I will try it on my wife's and daughters boxes, but they are on the same LAN here in the "office". Probably there are Internet Cafes in town that use the same ISP that we use at home. Note: 2 minutes ago, I was able to load the Home page of centos.org and also got into the Wiki. Lanny
On Fri, Mar 20, 2009 at 11:12 PM, John R Pierce pierce@hogranch.com wrote: <snip>
$ telnet www.centos.org www Trying 72.232.194.162... Connected to www.centos.org (72.232.194.162). Escape character is '^]'. HEAD / HTTP/1.1 Host: www.centos.org
HTTP/1.1 200 OK Date: Sat, 21 Mar 2009 04:08:16 GMT Server: Apache/2.0.52 (CentOS) X-Powered-By: PHP/4.3.9 Set-Cookie: PHPSESSID=d329988a9d984ee4bbe2e1e2f12d2aa0; path=/ Expires: Mon, 26 Jul 1997 05:00:00 GMT Cache-Control: private, no-cache Pragma: no-cache Connection: close Content-Type: text/html; charset=ISO-8859-1
Connection closed by foreign host.
note i had to send THREE lines... I used the HEAD command rather than GET so I don't have to stare at a bunch of html.
HEAD / HTTP/1.1 Host: www.centos.org
(and a blank line to terminate the request)
Last night, I reported that I was unable to get that to work on centos.org There was *no* response; but that it did work with my 2 web sites on servers in different DCs in CT. Last night, I could not load centos.org in my web browser.
At this time, I can load centos.org in my browser, without problems. Below are the results of 3 tests. Lanny
Test #1;
[lanny@dell2400 ~]$ telnet www.centos.org www Trying 72.232.194.162... Connected to www.centos.org (72.232.194.162). Escape character is '^]'. HEAD / HEAD/1.1 Host: www.centos.org HTTP/1.1 400 Bad Request Date: Sat, 21 Mar 2009 16:55:40 GMT Server: Apache/2.0.52 (CentOS) Connection: close Content-Type: text/html; charset=iso-8859-1
Connection closed by foreign host. [lanny@dell2400 ~]$
Test #2:
Connection closed by foreign host. [lanny@dell2400 ~]$ telnet www.centos.org www Trying 72.232.194.162... Connected to www.centos.org (72.232.194.162). Escape character is '^]'. HEAD / HEAD/1.1 Host: www.centos.org
HTTP/1.1 200 OK Date: Sat, 21 Mar 2009 17:09:23 GMT Server: Apache/2.0.52 (CentOS) X-Powered-By: PHP/4.3.9 Set-Cookie: PHPSESSID=93d50631f0343ff6e881c51badbba880; path=/ Expires: Mon, 26 Jul 1997 05:00:00 GMT Cache-Control: private, no-cache Pragma: no-cache Connection: close Content-Type: text/html; charset=ISO-8859-1
Connection closed by foreign host. [lanny@dell2400 ~]$
Test #3
When I run the command "mtr -c 10 -r centos.org" I continue to see the line which includes "???" and "100% loss" within the Layered Tech DC in Dallas. I don't know what if anything that indicates. At the time of the below test, I can load centos.org into my browser, without problems.
pod22c_ae.layeredtech.com 0.0% 120.8 136.4 118.6 201.5 26.3 ??? 100.0 0.0 0.0 0.0 0.0 0.0 162.194.232.72.static.reverse.ltdomains.com 10.0% 148.6 133.1 119.7 161.0 18.0 [root@dell2400 ~]#
<snip>
When I run the command "mtr -c 10 -r centos.org" I continue to see the line which includes "???" and "100% loss" within the Layered Tech DC in Dallas. I don't know what if anything that indicates. At the time of the below test, I can load centos.org into my browser, without problems.
pod22c_ae.layeredtech.com 0.0% 120.8 136.4 118.6 201.5 26.3 ??? 100.0 0.0 0.0 0.0 0.0 0.0 162.194.232.72.static.reverse.ltdomains.com 10.0% 148.6 133.1 119.7 161.0 18.0 [root@dell2400 ~]#
Not all routers respond to the icmp packets in these tests. The fact that the trace gets beyond that router is usually enough to imply that the packets get through. I also see the 100% loss at that router, but have no problem with the centos site.
Sometimes routers are configured to ignore tests like this all the time, and sometimes they will drop any unnecessary test packets if they are running near their maximum load. The tests are not a go / no go test.
2009/3/21 Scott Silva ssilva@sgvwater.com:
<snip> > When I run the command "mtr -c 10 -r centos.org" I continue to see the > line which includes "???" and "100% loss" within the Layered Tech DC > in Dallas. I don't know what if anything that indicates. At the time > of the below test, I can load centos.org into my browser, without > problems. > > pod22c_ae.layeredtech.com 0.0% 120.8 136.4 118.6 201.5 26.3 > ??? 100.0 0.0 0.0 0.0 0.0 0.0 > 162.194.232.72.static.reverse.ltdomains.com 10.0% 148.6 133.1 119.7 161.0 18.0 > [root@dell2400 ~]#
Not all routers respond to the icmp packets in these tests. The fact that the trace gets beyond that router is usually enough to imply that the packets get through. I also see the 100% loss at that router, but have no problem with the centos site.
Sometimes routers are configured to ignore tests like this all the time, and sometimes they will drop any unnecessary test packets if they are running near their maximum load. The tests are not a go / no go test.
Scott: Thank you for the explanation! I've been able to get to the centos.org web site today, without any problems, at various times during the day. Never had problems with it before, until this week. Lanny
2009/3/21 John R Pierce pierce@hogranch.com:
Lanny Marcus wrote:
HEAD / HEAD/1.1 Host: www.centos.org HTTP/1.1 400 Bad Request
its ...
HEAD / HTML/1.1
... not HEAD/1.1
I typed it wrong. Here's the example John Pierce gave me:
HEAD / HTTP/1.1 Host: www.centos.org
On Fri, Mar 20, 2009 at 11:07:03PM +0100, Ralph Angenendt wrote:
And all those plus the ones in the mails you and people before you sent do reach www.centos.org. What are you trying to prove here? Why is nobody talking to layered tech directly, if he/she cannot reach www.centos.org?
Additional datapoints showing that I can reach centos.org without issue from 4 locations (actually, quite a few more, these were the only ones I happened to cut-n-paste) geographically distributed around the continental US.
Plus my experience is that most providers don't care to receive third-party reports of problems that do not originate through their customers/clients directly.
John
Hi,
On Fri, Mar 20, 2009 at 02:20:47PM -0500, John R. Dennison wrote:
mtr from Philadelphia, PA, Level-3 outbound:
radagast.gerdesas.com Snt: 10 Loss% Last Avg Best Wrst StDev . 0.0% 0.1 0.1 0.1 0.1 0.0
...
ae-61-61.ebr1.Washington1.Level3.net 40.0% 11.1 9.9 4.2 16.3 4.8 ae-2.ebr3.Atlanta2.Level3.net 50.0% 17.9 20.8 17.9 24.3 3.0 ae-7.ebr3.Dallas1.Level3.net 60.0% 37.8 43.9 37.8 51.0 6.2 ae-83-83.csw3.Dallas1.Level3.net 0.0% 48.9 44.0 38.0 50.4 5.1 ae-32-89.car2.Dallas1.Level3.net 40.0% 38.0 97.1 38.0 222.9 75.7 DATABANK-HO.car2.Dallas1.Level3.net 0.0% 38.4 38.3 38.1 38.5 0.1 ae_cw_10g.databank.com 0.0% 38.4 38.5 38.1 38.9 0.2 pod22c_ae.layeredtech.com 0.0% 38.6 38.8 38.5 39.4 0.3 ??? 100.0 0.0 0.0 0.0 0.0 0.0 162.194.232.72.static.reverse.ltdomains.com 0.0% 38.6 38.5 38.3 38.7 0.1
mtr from Chicago, IL, NTT outbound:
HOST LOSS RCVD SENT BEST AVG WORST wrt1.beanproducts.net 0% 10 10 0.57 0.72 1.10
...
pod22f_ae.layeredtech.com 0% 10 10 30.59 32.21 41.55 ??? 100% 0 10 0.00 0.00 0.00 162.194.232.72.static.reverse.ltdomains.com 0% 10 10 30.75 31.11 31.43
mtr from Manchester, TN, Charter outbound:
tap.gerdesas.com Snt: 10 Loss% Last Avg Best Wrst StDev ??? 100.0 0.0 0.0 0.0 0.0 0.0
...
pod22f_ae.layeredtech.com 0.0% 92.4 80.2 53.5 117.8 22.8 ??? 100.0 0.0 0.0 0.0 0.0 0.0 162.194.232.72.static.reverse.ltdomains.com 0.0% 54.1 69.6 53.4 136.7 29.1
mtr from Boston, MA, Level-3 outbound:
HOST: neural Loss% Snt Last Avg Best Wrst StDev
- 2n+1-m160-1.optr.net 0.0% 10 0.3 0.4 0.3 0.5 0.1
...
- ae-2-7.bar1.Boston1.Level3.n 0.0% 10 0.3 9.8 0.3 95.2 30.0
- ae-0-11.bar2.Boston1.Level3. 0.0% 10 0.3 0.7 0.3 4.4 1.3
- ae-8-8.ebr1.NewYork1.Level3. 0.0% 10 5.0 5.0 5.0 5.1 0.0
- ae-91-91.csw4.NewYork1.Level 0.0% 10 13.0 9.7 5.3 17.4 4.3
- ae-94-94.ebr4.NewYork1.Level 0.0% 10 10.3 10.8 5.8 18.7 4.7
- ae-3.ebr2.Dallas1.Level3.net 0.0% 10 40.1 44.0 39.9 52.9 4.7
- ae-62-62.csw1.Dallas1.Level3 0.0% 10 40.2 46.2 40.2 53.3 4.5
- ae-12-69.car2.Dallas1.Level3 0.0% 10 40.4 42.9 40.3 64.4 7.5
- DATABANK-HO.car2.Dallas1.Lev 0.0% 10 40.9 40.6 40.3 40.9 0.2
- aw_cw_10g.databank.com 0.0% 10 41.0 40.6 40.3 41.0 0.2
- pod22a_aw.layeredtech.com 0.0% 10 41.0 41.1 40.6 43.0 0.7
- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
- 162.194.232.72.static.revers 0.0% 10 41.0 40.7 40.4 41.0 0.2
thanks for the info. From my limited understanding, it's NOT within Layered Tech infrastructure which is hosting www.centos.org, but one some routers (not all) managed by Level3.
Tru
-
Flaherty, Patrick -
John R Pierce -
John R. Dennison -
JohnS -
Lanny Marcus -
Marcelo Roccasalva -
Per Qvindesland -
R P Herrold -
Rainer Duffner -
Ralph Angenendt -
Robert -
Ross Walker -
Scott Silva -
Tru Huynh -
Victor Padro -
William L. Maltby