Practical limit to the size of a hosts file?

[Oober_Noober]

I've recently pumped up my hosts file with several different collections of loopback redirects of less-than-desireable sites. My hosts file is now hovering around 2MB and may grow a little bit more. I've searched around, but can't seem to find much readily available information about a practical limit to a hosts file's size. Heres the only significant link I came accross: http://www.techsupportalert.com/issues/issue97.htm (Just search for the word "Personally" on the page and you'll come to the paragraph reference hosts file size). As you'll see, no hard facts are presented for the writer's views on the hosts file.

I'm wondering if having so many entires would bog down the name resolution process. I've been having some problems lately with internet connectivity, but alot of it seems to be related to my router so its difficult to pinpoint if my hosts file could be adding to the problem.

Any thoughts?


Thanks in advance,
--Oober

[Rob Pomeroy]

I'm wondering if having so many entires would bog down the name resolution process.

Without a doubt.  An intelligent hardware firewall will provide a much more efficient solution.  Possibly your router is up to this job.

[Raptor]

Are there routers that can do that?

I think it's cheaper and easier to just have your host file grow out of proportions, mine is 494 KB and I never noticed any slowdowns.

[Oober_Noober]

I'm wondering if having so many entires would bog down the name resolution process.

Without a doubt.  An intelligent hardware firewall will provide a much more efficient solution.  Possibly your router is up to this job.

I doubt it. It's a Linksys WRT54G Wireless-G Router which has enough features to be fun to play with, but isn't as much fun as something rack mounted. What always made me curious about the hosts file is: why is the IP address first? I would think that the hostname would be first followed by an IP address to map it to. In my mind, that would make it easier for the computer to parse through the text file. Right now there are thousands of "127.0.0.1" entries it has to look through before coming to the hostname.

Raptor, my hosts file is a little over 2,000 KBytes. I've attached it to this post. Its actually in two parts to circumvent the 250k attachment limitation. Control-c and control-v work nicely. You can try it for yourself and see if it causes problems.

Here is an intersting page about large hosts files, the problem that they can allegedly cause, and a purported fix: http://www.ericphelps.com/scripting/samples/Hosts/ In the article, it is suggested that you disable the DNS client service. Sounds interesting. I'll give it a try when I have a spare minute or three. In the meantime, does anyone else have any leads on hosts file size?

Thanks,.
--Oober

[Oober_Noober]

Hosts file part 2

--O.N.

[Raptor]

How'd you go about generating all that?

I'm still waiting for a Firefox extension that will allow me to write any adress I connect to to host file. =)

[Oober_Noober]

How'd you go about generating all that?

I'm still waiting for a Firefox extension that will allow me to write any adress I connect to to host file. =)

I just Googled around to find some good anti-malware hosts files and found about three. Copy and Paste is a wonderful invention. Here's a link to one of the largest such hosts files: http://www.mvps.org/winhelp2002/hosts.txt It seems to stay very well updated. In fact, its probably the only one I really need, but I thought I'd be *really* thorough and add a few together. I still haven't had a chance at trying the aforementioned disabling of the DNS client service to see if it clears my problems up. BTW, the problem I was having was that I would type a hostname into my browser (apple.com, download.com, whatever) and it would just hang there. However, I've been having simultaneous router problems, so I can't be sure what the source of the problem is.

As far as that Firefox hack you were wishing for, I'm sure it wouldn't be too hard with a little programming knowledge. Mention it on the Mozilla forums and you may get a response. Or you could always start hacking away at the firefox source code on your own (http://developer.mozilla.org/en/docs/Download_Mozilla_Source_Code).


--Oober

[Raptor]

Here's a link to one of the largest such hosts files: http://www.mvps.org/winhelp2002/hosts.txt /quote]

I use that one as well. I keep being surprised at how many ads people make comments on on forums and I'm thinking Advertisements? I don't see any!

I'm not too keen on programming, but I might pick it up one day and see if I can get anything done. I once suggested it to a programmer and he said you might need to build drivers of some kind because you're influencing something outside of Firefox.

However, I've been having simultaneous router problems, so I can't be sure what the source of the problem is.

Funny enough is that I'm also having a bit of trouble with my Router and it happens to be a WRT54G Wireless-G Router  as well.

I sometimes lose connection for a second or so and then it reconnects; oddly enough my Qtec router never had a problem with that.

Although it may have just been the cabling that got jerked loose when I placed my new router; I yet have to see the error/disconnect message after reconnecting the cables.  

What problems are you having with that Router?

[Oober_Noober]

I use that one as well. I keep being surprised at how many ads people make comments on on forums and I'm thinking Advertisements? I don't see any!

Yeah, adblock plus and filterset G also keep me ad-free.

Funny enough is that I'm also having a bit of trouble with my Router and it happens to be a WRT54G Wireless-G Router  as well. I sometimes lose connection for a second or so and then it reconnects; oddly enough my Qtec router never had a problem with that. Although it may have just been the cabling that got jerked loose when I placed my new router; I yet have to see the error/disconnect message after reconnecting the cables.  
 
What problems are you having with that Router?  

It'll just spontaneously stop accepting any traffic whatsoever. ipconfig /all shows everything in order, but no traffic is moved. Not even ICMP. Sometimes ipconfig /renew solves the problem, other times you've got to "bounce" it (unplug the power jack, wait 10 or 15 seconds, and then plug it back in). Weird.

--O.N.

[Rob Pomeroy]

You might get an answer on MSDN about the theoretical limit of hsots files.

Note that many routers are now based on Linux, providing an extremely efficient and highly configurable router/firewall/packet analyser combo.  You could drop one of your hosts files into such a router, and I expect you'd get much better results.  Or you could set up all kinds of wildcard/IP address range filtering.

Bottom line, right tool for job.  If you need something that AdblockPlus and a decent software firewall can't cope with, then get a decent hardware firewall.

IMHO.