Pete, we initially did use an ACL, but that's such a heavy hammer we want
to avoid that.

Besides reading the Listserv logs, I dug into the iis logs, and also used
Google's tools to find out what they say they are doing. Bottom line: I
now believe they are doing what they are supposed to, and not doing extra
scanning. If every message is a separate web page, and you have a few very
busy lists, the number of pages to crawl gets large. What puzzled me
initially (and still does), is why the hits in the Listserv logs went up
so precipitously sometime during February, from a few per day to 10's of
thousands ( Google says they average 50k pages/day).

My public list owners want their archives on the web, so I guess this is
the price to pay. If needed, it appears that I can reduce the frequency
somewhat. Assuming I could figure out how to do what Andrew did, I am
reluctant to stop scanning the older archives even if they remained in
Google's records since we sometimes move in lists from other sites.

Server cpu usage hasn't been horrible despite the many hits. Although I
initially thought our Listserv failure might have been due to the high
numbers of hits, I now doubt there's any correlation. My conclusion was
based on the notion this was a new condition, and while I verified that it
was "new" since February, it wasn't brand new. So I don't know why
Listserv got ill- it happens so seldom that we don't know what to do:-)

If anyone has further suggestions, I'm open to ideas or to be convinced
I'm wrong. I really, really appreciate all of the help I've received.
Thanks to Andrew Bosch, Andy Smith-Peterson, Nathan Brindle, Pete Weiss,
Michael Shannon, and others for their invaluable suggestions.

Nelson

-- Syracuse University Listserv List Manager
-- Listserv webpage: http://listserv.syr.edu