[CentOS] Centos 4, bind slow responses when no cache

I have a Centos 4.0, bind-9.2.4-2 as a master and and FC2 bind-9.2.3-13
slave setup.
Their seem to be a delay on the Centos 4 when asking for anything that
is not in its cache. Doing some tcpdumps I see the AUTH response came
back, but like a 3 second delay after that before server sends response
to client. Both server have the same config other than one being a
master and the other a slave. Their are no delays on the FC2.

Googled on this with no joy.

Any ideas?


Ted