Please refer to my thread "excessive DNS slows httpd" in which I discuss a very slow response from my httpd server on my CentOS box after modifying httpd.config .
I have "solved" the problem by blocking all DNS in iptables on my CentOS box. As it happens, this is not a problem in my present configuration, but it can hardly be considered a good solution. Given that the httpd configurations are essentially identical, it appears that there is a difference between my Feodora box and my CentOS box regarding how and when DNS is handled, at least when originated by httpd.
Suggestions would be most welcome.
Mike.
Mike -- EMAIL IGNORED wrote on Tue, 4 Aug 2009 16:56:22 +0000 (UTC):
Please refer to my thread "excessive DNS slows httpd"
Why don't you keep posting in there then?
Suggestions would be most welcome.
Foremost, you want to find out why those queries are generated despite the fact that hostnamelookups are off. That is surely something in your configuration or a web application. It's not a general problem of Apache on CentOS, mine are all not doing that.
Kai
On Tue, 04 Aug 2009 21:31:22 +0200, Kai Schaetzl wrote:
Mike -- EMAIL IGNORED wrote on Tue, 4 Aug 2009 16:56:22 +0000 (UTC):
Please refer to my thread "excessive DNS slows httpd"
Why don't you keep posting in there then?
Because the new title reflects a new focus, and I hoped to attract different people.
Suggestions would be most welcome.
Foremost, you want to find out why those queries are generated despite the fact that hostnamelookups are off. That is surely something in your configuration or a web application. It's not a general problem of Apache on CentOS, mine are all not doing that.
Kai
Until now, for a long time, mine wasn't doing it either.
Part of the problem was suggested by someone on the Apache group; there are two problems:
1. Both boxes have nscd, but it was not running on the CentOS box. Now that I fixed that, all but the first connection are rapid, as you might expect.
2. We still do not why the change in httpd.conf caused the problem to appear. However, my belief that there was a difference between the two machines is accounted for by the difference in /etc/init.d/nscd . When I tried it for the first time this morning, the box that previously been fast was slow. No doubt, the nscd storage had timed out.
Except that nscd was not set to run, it is probably not specifically a CentOS problem. Perhaps I made a wrong choice in setup?
Mike.
Mike -- EMAIL IGNORED wrote:
Except that nscd was not set to run, it is probably not specifically a CentOS problem. Perhaps I made a wrong choice in setup?
Something I like to do when troubleshooting apache related things is turn on mod_status and mod_info and compare the results there(mod_info really mod_status for realtime stuff), just in the off chance a config is loading that you don't expect, or perhaps apache wasn't properly restarted/reloaded since the last change etc.
nate
On Tue, 04 Aug 2009 14:58:17 -0700, nate wrote:
Mike -- EMAIL IGNORED wrote:
Except that nscd was not set to run, it is probably not specifically a CentOS problem. Perhaps I made a wrong choice in setup?
Something I like to do when troubleshooting apache related things is turn on mod_status and mod_info and compare the results there(mod_info really mod_status for realtime stuff), just in the off chance a config is loading that you don't expect, or perhaps apache wasn't properly restarted/reloaded since the last change etc.
nate
I have good reason to believe that the loaded config is largely correct; do think that this can tell me why there are DNS calls?
Thanks, Mike.
Mike -- EMAIL IGNORED wrote:
On Tue, 04 Aug 2009 21:31:22 +0200, Kai Schaetzl wrote:
Mike -- EMAIL IGNORED wrote on Tue, 4 Aug 2009 16:56:22 +0000 (UTC):
Please refer to my thread "excessive DNS slows httpd"
Why don't you keep posting in there then?
Because the new title reflects a new focus, and I hoped to attract different people.
Suggestions would be most welcome.
Foremost, you want to find out why those queries are generated despite the fact that hostnamelookups are off. That is surely something in your configuration or a web application. It's not a general problem of Apache on CentOS, mine are all not doing that.
Kai
Until now, for a long time, mine wasn't doing it either.
Part of the problem was suggested by someone on the Apache group; there are two problems:
Both boxes have nscd, but it was not running on the CentOS box. Now that I fixed that, all but the first connection are rapid, as you might expect.
We still do not why the change in httpd.conf caused the problem to appear. However, my belief that there was a difference between the two machines is accounted for by the difference in /etc/init.d/nscd . When I tried it for the first time this morning, the box that previously been fast was slow. No doubt, the nscd storage had timed out.
Except that nscd was not set to run, it is probably not specifically a CentOS problem. Perhaps I made a wrong choice in setup?
If I really want to know what is different between two boxes, I'll do something like NFS mount one into the other or rsync their /etc trees somewhere on a common host and let diff -r walk through them.
Are you sure this isn't as simple as having (and using, check your resolv.conf) a caching name server running on one box so most lookups are resolved locally while the other is making the query to something slow?
On Tue, 04 Aug 2009 17:23:04 -0500, Les Mikesell wrote:
Mike -- EMAIL IGNORED wrote:
On Tue, 04 Aug 2009 21:31:22 +0200, Kai Schaetzl wrote:
[...]
Until now, for a long time, mine wasn't doing it either.
Part of the problem was suggested by someone on the Apache group; there are two problems:
Both boxes have nscd, but it was not running on the CentOS box. Now that I fixed that, all but the first connection are rapid, as you might expect.
We still do not why the change in httpd.conf caused the problem to appear. However, my belief that there was a difference between the two machines is accounted for by the difference in /etc/init.d/nscd . When I tried it for the first time this morning, the box that previously been fast was slow. No doubt, the nscd storage had timed out.
Except that nscd was not set to run, it is probably not specifically a CentOS problem. Perhaps I made a wrong choice in setup?
If I really want to know what is different between two boxes, I'll do something like NFS mount one into the other or rsync their /etc trees somewhere on a common host and let diff -r walk through them.
As indicated above, I already know the difference between the two boxes.
Are you sure this isn't as simple as having (and using, check your resolv.conf) a caching name server running on one box so most lookups are resolved locally while the other is making the query to something slow?
resolve.config is the same on the two boxes.
Mike.
Mike -- EMAIL IGNORED wrote:
If I really want to know what is different between two boxes, I'll do something like NFS mount one into the other or rsync their /etc trees somewhere on a common host and let diff -r walk through them.
As indicated above, I already know the difference between the two boxes.
I find that a little hard to believe, since they behave differently and you seem not to understand why.
Are you sure this isn't as simple as having (and using, check your resolv.conf) a caching name server running on one box so most lookups are resolved locally while the other is making the query to something slow?
resolve.config is the same on the two boxes.
Assuming you mean resolv.conf, is the the first choice one of the boxes in question?
On Tue, 04 Aug 2009 18:28:13 -0500, Les Mikesell wrote:
Mike -- EMAIL IGNORED wrote:
If I really want to know what is different between two boxes, I'll do something like NFS mount one into the other or rsync their /etc trees somewhere on a common host and let diff -r walk through them.
As indicated above, I already know the difference between the two boxes.
I find that a little hard to believe, since they behave differently and you seem not to understand why.
I have corrected the difference, and they now behave the same.
Are you sure this isn't as simple as having (and using, check your resolv.conf) a caching name server running on one box so most lookups are resolved locally while the other is making the query to something slow?
resolve.config is the same on the two boxes.
Assuming you mean resolv.conf, is the the first choice one of the boxes in question?
Yes, resolv.conf , I am not sure that I understand the question. The resolv.conf for the two boxes are identical. Each contain two router addresses. I have two nested routers, and each have the same two real DNS addresses.
Mike.
Mike -- EMAIL IGNORED wrote:
On Tue, 04 Aug 2009 18:28:13 -0500, Les Mikesell wrote:
Mike -- EMAIL IGNORED wrote:
If I really want to know what is different between two boxes, I'll do something like NFS mount one into the other or rsync their /etc trees somewhere on a common host and let diff -r walk through them.
As indicated above, I already know the difference between the two boxes.
I find that a little hard to believe, since they behave differently and you seem not to understand why.
I have corrected the difference, and they now behave the same.
I thought you said you blocked something with iptables.
Are you sure this isn't as simple as having (and using, check your resolv.conf) a caching name server running on one box so most lookups are resolved locally while the other is making the query to something slow?
resolve.config is the same on the two boxes.
Assuming you mean resolv.conf, is the the first choice one of the boxes in question?
Yes, resolv.conf , I am not sure that I understand the question. The resolv.conf for the two boxes are identical. Each contain two router addresses. I have two nested routers, and each have the same two real DNS addresses.
It is common to run a caching nameserver to reduce the traffic for lookups. I thought perhaps you had them configured on both machines but used the same one as the first choice.
On Tue, Aug 4, 2009 at 5:54 PM, Mike -- EMAIL IGNOREDm_d_berger_1900@yahoo.com wrote:
On Tue, 04 Aug 2009 21:31:22 +0200, Kai Schaetzl wrote:
Mike -- EMAIL IGNORED wrote on Tue, 4 Aug 2009 16:56:22 +0000 (UTC):
Please refer to my thread "excessive DNS slows httpd"
Why don't you keep posting in there then?
Because the new title reflects a new focus, and I hoped to attract different people.
Suggestions would be most welcome.
Foremost, you want to find out why those queries are generated despite the fact that hostnamelookups are off. That is surely something in your configuration or a web application. It's not a general problem of Apache on CentOS, mine are all not doing that.
Kai
Until now, for a long time, mine wasn't doing it either.
Part of the problem was suggested by someone on the Apache group; there are two problems:
- Both boxes have nscd, but it was not running on the CentOS
box. Now that I fixed that, all but the first connection are rapid, as you might expect.
- We still do not why the change in httpd.conf caused the
problem to appear. However, my belief that there was a difference between the two machines is accounted for by the difference in /etc/init.d/nscd . When I tried it for the first time this morning, the box that previously been fast was slow. No doubt, the nscd storage had timed out.
Except that nscd was not set to run, it is probably not specifically a CentOS problem. Perhaps I made a wrong choice in setup?
If you are uncertain where the DNS lookups are originating from, you could always add an interposer between Apache and glibc. When the culprit calls one of the get* routines, you could log a backtrace to a file. Summarizing the backtraces should lead you to the modules (and directives) responsible for the lookups. If you happen to have a Solaris 10 / OpenSolaris host handy, you can use the DTrace ustack() action to do just this. No need to muck around with interposing on the glibc calls.
Hope this helps, - Ryan -- http://prefetch.net
On Tue, 04 Aug 2009 18:55:01 -0400, Matty wrote:
[...]
If you are uncertain where the DNS lookups are originating from, you could always add an interposer between Apache and glibc. When the culprit calls one of the get* routines, you could log a backtrace to a file. Summarizing the backtraces should lead you to the modules (and directives) responsible for the lookups. If you happen to have a Solaris 10 / OpenSolaris host handy, you can use the DTrace ustack() action to do just this. No need to muck around with interposing on the glibc calls.
Hope this helps,
- Ryan
I don't have Solaris; where would I get the interposer, and how would I add it?
Mike.
Am Mittwoch, den 05.08.2009, 00:55 +0200 schrieb Matty:
If you happen to have a Solaris 10 / OpenSolaris host handy, you can use the DTrace ustack() action to do just this. No need to muck around with interposing on the glibc calls.
Whe have the same it's called systemtap....
financial.com AG
Munich head office/Hauptsitz München: Maria-Probst-Str. 19 | 80939 München | Germany Frankfurt branch office/Niederlassung Frankfurt: Messeturm | Friedrich-Ebert-Anlage 49 | 60327 Frankfurt | Germany Management board/Vorstand: Dr. Steffen Boehnert (CEO/Vorsitzender) | Dr. Alexis Eisenhofer | Dr. Yann Samson | Matthias Wiederwach Supervisory board/Aufsichtsrat: Dr. Dr. Ernst zur Linden (chairman/Vorsitzender) Register court/Handelsregister: Munich – HRB 128 972 | Sales tax ID number/St.Nr.: DE205 370 553
On Wed, Aug 5, 2009 at 4:21 AM, Christoph Masercmr@financial.com wrote:
Am Mittwoch, den 05.08.2009, 00:55 +0200 schrieb Matty:
If you happen to have a Solaris 10 / OpenSolaris host handy, you can use the DTrace ustack() action to do just this. No need to muck around with interposing on the glibc calls.
Whe have the same it's called systemtap....
Systemtap is super useful, but unfortunately it doesn't do userland tracing. Hence the reference to the DTrace ustack() action.
Thanks for the feedback, - Ryan -- http://prefetch.net
-
Christoph Maser -
Kai Schaetzl -
Les Mikesell -
Matty -
Mike -- EMAIL IGNORED -
nate