Try this.
Place a robots.txt file into the root of the web server with this text in it. Asterisks aren't in the file.
*************************************
# robots.txt for http://yourservernamehere
User-agent: *
Disallow: /
*************************************
This will stop all spiders and bots from searching anything on the server. The ones that respect the robots.txt file.
Search on google for robots.txt if you want to customize it.
--
Nate Eckstine (DoIT CityWide Webteam)
City of Seattle Department of Information Technology
700 Fifth Ave. Suite 2700
Seattle, WA 98124-4709 USA
Work (206) 733-9658
Fax (206) 684-0911
>>> Darren Evans-Young <[log in to unmask]> 10/31/2007 2:18 PM >>>
I just happened to notice in my logs a number X-LOGCHK entries. 48,000 to
be exact, since midnight. I traced the IP to Google. They are beating on
my server and I dont like it. What is the best way to stop it?
Others may want to check their logs too.
Darren Evans-Young
Univ of Alabama
