Yes Google for some time has been able to index dynamically created
content including those of various content management systems as well as
Listserv archives.  I think this became a mandatory development issue
when they got into Froogle the commerce catalog engine requirements.

Experience at my site is that it can/has indexed publicly open archives
and not private ones. Our private lists are not featured on the site
list index and ONLY indexes will show if you search for the list by
exact name.  This Google does not know and will not index UNLESS someone
specifically submits it in the form;
http://domain.com/archives/wa.cgi?LIST=listname . Even then Gogglebots
can not crawl/index any further the the index to the protected messages
themselves even if it wanted to.

Regards,
Omer.



> -----Original Message-----
> From: LISTSERV site administrators' forum
> [mailto:[log in to unmask]] On Behalf Of Stephen C. Nill
> Sent: Thursday, May 06, 2004 3:47 PM
> To: [log in to unmask]
> Subject: Google, the Deep Web, and LISTSERV Archives
>
>
> [Cross-posted to LSTSRV-L and LSTOWN-L.]
>
> Colleagues,
>
> I've checked the archives of LSTSRV-L and LSTOWN-L and
> believe this is the first time this has been discussed....
>
> Google now appears to be able to search what has been termed
> the "deep web," meaning that it is able to access and pull
> data from dynamically-generated web pages such as LISTSERV archives.
>
> We discovered this when we began to receive requests from
> list subscribers to remove their postings from our archives.
> They explained that when they Google their names (yes, "to
> Google" is now a verb), their postings come up and often lead
> the list of search results.
>
> It appears preliminarily that only those LISTSERV lists that
> provide publicly-available archives are reached by Google.
> I'm interested in confirming this aspect with my colleagues
> here.  Are only those archives set for public access by
> subscribers and nonsubscribers searchable by Google?
>
> We are considering the policy implications of this apparent
> new ability of Google.  While we have always made our public
> list archives fully available to subscribers and
> non-subscribers alike, and believe that to be important to
> the nonprofit sector that we serve, we may conclude that the
> usefulness of public archives is outweighed by the possibly
> chilling effect on discussions by nonprofit professionals who
> understandably may now be hesitant to "talk shop" if a donor
> can find such discussions in seconds by Googling them.  On
> the other hand, we may conclude that Google will now bring
> our archives, which comprise perhaps the largest body of
> discussion about best practices in the nonprofit sector in
> the world, into reach of more nonprofit, or NGO,
> organizations.  It's an important issue for us and I'm sure
> for others who maintain LISTSERV and/or who are list owners.
>
> As this seems to be an emerging issue not heretofore
> discussed here, I'm quite interested in how my colleagues
> here are dealing with this.
>
> Steve
>
>
>
> Stephen C. Nill, J.D.
> Founder & Chief Executive Officer [log in to unmask]
>
> CharityChannel.com - The Conference That Never Sleeps (tm)
>