> It turns out the problem was too many files in the listserv\main
> directory. Not too many lists.
>
> We enabled content_filter a while back - moderating any message that
> our spam tagging system designated as spam. Most of the list owners
> were just ignoring the messages about approval so the .OK files were
> just hanging around. 29000 of them.

This is probably due to the infamous 8.3 filename emulation in Windows, which is necessary if you want to run 16-bit applications. Few people still use 16-bit code, but the feature is enabled by default and it can have a huge performance impact. Unfortunately, turning it off can sometimes break third-party applications even though they no longer run in 16-bit mode. It is a gamble on a multi-purpose server, but you can usually do it without problem if LISTSERV is the only thing you run on the machine.

To confirm that this is the problem, do a DIR /X on the directory where you moved off the offending files. You will see entries like the following:

2008-02-24  18:28            13,228 LSOFT-~2.OK1 LSOFT-EU-NEWSLETTER.OK18E5B5
2008-02-24  19:55             3,366 LSOFT-~1.OK3 LSOFT-EU-NEWSLETTER.OK3DAD29
2008-02-24  21:09             7,065 LSOFT-~4.OK4 LSOFT-EU-NEWSLETTER.OK469CB4
2008-02-24  09:27            10,589 LSOFT-~3.OK4 LSOFT-EU-NEWSLETTER.OK470CC1
2008-02-24  00:24            10,188 LSOFT-~2.OK4 LSOFT-EU-NEWSLETTER.OK487B32
2008-02-23  23:11             2,728 LSOFT-~3.OK5 LSOFT-EU-NEWSLETTER.OK544491
(...)

The emulated 8.3 filename comes first, then the real filename. This is an example where there is no noticeable slowdown because the number after the '~' remains small. As the number of files matching the same 8.3 pattern increases, Windows has to loop through more and more numbers to find one that is not in use. A larger number also "eats" into the prefix to the left of the '~' sign, making it smaller and less differentiating, which increases the number of conflicts and leads to even larger numbers. What I mean is, once you reach let's say 100 files with the same pattern, you no longer have names like 'LSOFT-~4.OK4' but 'LSOF~123.OK4'. There are even more conflicts with other lists and thus even higher numbers, and soon enough you find yourself with files called 'LSO~1234.OK4'. Many organizations have lists named after 3-4 letter department codes, and you may end up with all your lists mapped to the same prefix. The code in Windows that creates these 8.3 names is extremely inefficient and scales in n2.

For permanent relief, you can disable this through a registry entry. See:

http://support.microsoft.com/kb/121007

NTFS can actually handle very large directories as long as you disable the 8.3 feature. It was the fastest file system for LISTSERV when introduced 14 years ago, nowadays this is no longer the case but it remains one of the top performers, but only if you disable this.

  Eric