My understanding of the file is that it has to be in the root directory of the webserver.

--
Nate Eckstine (DoIT)

City of Seattle Department of Information Technology
700 Fifth Ave. Suite 2700
Seattle, WA 98124-4709   USA

Work (206) 733-9658
Fax (206) 684-0911

 


>>> Darren Evans-Young <[log in to unmask]> 10/31/2007 2:30 PM >>>
On Wed, 31 Oct 2007, Nate Eckstine wrote:

>Try this.
>
>Place a robots.txt file into the root of the web server with this text in it. Asterisks aren't in the file.
>
>*************************************
>
># robots.txt for http://yourservernamehere 
>
>User-agent: *
>Disallow: /
>
>*************************************
>
>This will stop all spiders and bots from searching anything on the server. The ones that respect the robots.txt file.
>
>Search on google for robots.txt if you want to customize it.
>
>
>--
>Nate Eckstine (DoIT CityWide Webteam)

I figured something like that would work. But I dont have access to the
web server doc root. Would putting this in Listserv's archives directory
prevent their bot from searching just that directory (and down)?

Darren