On: Wed, Oct 26, 2005 at 04:08:06PM -0500,Chris Simon Wrote:

} Has anyone seen a problem with their server in regards to a Google crawler?
} The crawler has been going through our archives about twice a week causing
} high CPU usage for an hour or so.  It has been happening on off-hours so it
} hasn't had a huge impact but I'm curious to see if anyone else has
} experienced this.

Put a "robots.txt" file in the directory (or the root).  Most spiders
will honor it.  You can get the format by googling.

I think the following is a "keep out of everything:

     User-agent: *
     Disallow: / 

-- 
Rich Greenberg N6LRT Marietta, GA, USA richgr atsign panix.com + 1 770 321 6507
Eastern time zone.   I speak for myself & my dogs only.     VM'er since CP-67
Canines:Val, Red & Shasta (RIP),Red, husky                   Owner:Chinook-L
Atlanta Siberian Husky Rescue. www.panix.com/~richgr/   Asst Owner:Sibernet-L