LSTSRV-L Archives

LISTSERV Site Administrators' Forum

LSTSRV-L
Options: Use Monospaced Font
Show Text Part by Default
Condense Mail Headers

Topic: [<< First] [< Prev] [Next >] [Last >>]

Parts/Attachments: text/plain (45 lines)
Print Reply
Mime-Version:
1.0
Content-Type:
text/plain; charset=US-ASCII
Date:
Wed, 31 Oct 2007 14:33:46 -0700
Content-Disposition:
inline
Reply-To:
LISTSERV give-and-take forum <[log in to unmask]>
Subject:
Re: Google
From:
Nate Eckstine <[log in to unmask]>
In-Reply-To:
<[log in to unmask]>
Content-Transfer-Encoding:
quoted-printable
Sender:
LISTSERV give-and-take forum <[log in to unmask]>
 
My understanding of the file is that it has to be in the root directory of the webserver.

--
Nate Eckstine (DoIT)

City of Seattle Department of Information Technology
700 Fifth Ave. Suite 2700
Seattle, WA 98124-4709   USA

Work (206) 733-9658
Fax (206) 684-0911

 


>>> Darren Evans-Young <[log in to unmask]> 10/31/2007 2:30 PM >>>
On Wed, 31 Oct 2007, Nate Eckstine wrote:

>Try this.
>
>Place a robots.txt file into the root of the web server with this text in it. Asterisks aren't in the file.
>
>*************************************
>
># robots.txt for http://yourservernamehere 
>
>User-agent: *
>Disallow: /
>
>*************************************
>
>This will stop all spiders and bots from searching anything on the server. The ones that respect the robots.txt file.
>
>Search on google for robots.txt if you want to customize it.
>
>
>--
>Nate Eckstine (DoIT CityWide Webteam)

I figured something like that would work. But I dont have access to the
web server doc root. Would putting this in Listserv's archives directory
prevent their bot from searching just that directory (and down)?

Darren

ATOM RSS1 RSS2