RE: DNS lookup problems after upgrade from Etch to Lenny
>From: Stackpole, Chris [mailto:CStackpole@barbnet.com]
>Sent: Monday, June 15, 2009 4:36 PM
>Subject: RE: DNS lookup problems after upgrade from Etch to Lenny
>
>> From: Stackpole, Chris [mailto:CStackpole@barbnet.com]
>> Sent: Monday, June 15, 2009 4:30 PM
>> Subject: DNS lookup problems after upgrade from Etch to Lenny
>>
>> Hey guys,
>>
>> I have been trying my best to figure this out recently but not having
>> much luck. I am hoping someone here might be able to help.
>>
>> We had an Etch box that had been running great for almost 2 years.
>> When we decided that we really should upgrade to Lenny, the box had
>> over 300 days of uptime. It was a really solid box that handled a ton
>> of data for us.
>>
>> There are a number of reasons, but we needed to update the system. So
>> I ran the upgrade process. I did an `apt-get update` and `apt-get
>> upgrade` on Etch, restarted, changed out sources to Lenny then did an
>> `apt-get dist-upgrade`. Standard and simple and everything seemed to
go
>> really well.
>>
>> However, shortly after, it started slamming our DNS servers. It went
>> from an average of 3.5 requests per minute to over 7000 requests per
>> minute. Every time it talks to a system, it does a DNS lookup.
>
> Scratch that bit. My network guy just sent me the official numbers. It
> jumped from 3.5 requests per minute to 73,955.25 requests per minute.
A
> factor of 10 worse. :-0
>
>> At first I thought it was Apache and or the applications we run, but
>> after some testing, I don't think that is the case. I stopped all the
>> programs and apache before I started pinging other systems by their
>> domain name. It constantly asked the DNS for information.
>>
>> How do I know? I am running `tshark -f 'port 53'` and watching all
the
>> data in real time. When I ping a name, it does a lookup. If I leave
>> the ping running, after a few seconds it asks again. I set up a loop
>> using `ping -c1` and no matter how fast the loop ran, it asked for
the
>> name every time.
>>
>> So a busy server working with ~50 other computers and tons of
>> connections is asking the DNS for every connection!
>>
>> Can any one help out? I am digging around in all sorts of conf files
>> but not finding anything. My search online seems to suggest I should
>> install a DNS caching utility, but I don't understand why I have this
>> problem.
>>
>> I didn't have it before and on another system running a fresh build
of
>> Lenny running the exact same software handling a similar load and its
>> DNS ask rate is just as low as it was on this system running Etch.
>>
>> I am at a loss. I appreciate any help people can give.
>>
>> Thanks,
>> Chris Stackpole
Well I can't seem to find and fix this problem. So I guess my two
options
are: 1) rebuild the system using Lenny 2) Install a DNS caching utility.
Right now I am trying number 2. I installed dbndns and tied to configure
it
using this [1] talk as a guide, but I think I goofed something up. It
isn't
working.
Can anyone help out in configuring this?
Thanks,
Chris Stackpole
[1] http://bevilacqua.us/talks/djbdns/
Reply to: