These are *JOB* files that listserv has not processed yet. We run 18 smtp helpers on the outgoing queues and that seems to keep up just fine.  We do *not* run spam assassin or any other filters.  As I said, they would just make our input queues worse, in my opinion.  They would save our readers from looking at spam, so we might do this someday. 

I don't know why this bottleneck happened although the queues have been getting worse over the last several months.  It is interesting, in a good way, that a lot of work flows paste all of these queued jobs.  A "Thanks" message often comes right back, and  a lot of list traffic gets through.  Listserv takes a "quick" look at incoming files and tries to prioritize the job queue in some fashion.

From Monday's logs, I gleaned the following numbers:

67,000 incoming "jobs" based on "Processing file" lines
12,700 files rejected (probably spam) with message you are not allowed to post to this list.
10,500 rejected as coming from accounts that had been "served off drop"
 6,500 x-spam jobs from other server flagging spammers
 4,000 file asked for confirmation - probably most to -request addresses
 4,000 files that were actually mail to some lists.
Note that the last 5 items were included in the first item.

At 7:08 PM -0400 4/18/06, Valdis Kletnieks wrote
>On Tue, 18 Apr 2006 18:54:38 EDT, Bill Verity said:
>
>> However, I think the system is running fine.  We processed 67,000
>> files yesterday which is a lot but not a record.  As far as I can
>> tell, more than 3/4 of that load was garbage.
>
>Can you give a better description of what you were seeing than "3/4 was garbage"?
>
>Do you have any idea where the bottleneck was, exactly?  Between the sending
>box and your Listserv's MTA?  Sitting in the Listserv queue? Listserv waiting
>to hand off the mail to the outbound MTA (SMTP_FORWARD_1="3*localhost" was a
>big win for us there, as was 'FEATURE(nocanonify)' in sendmail.mc).
>
>> If we decide to run spamassasin, that will just require more hardware
>> without stemming the flow of incoming junk.
>
>If you're running spamassassin, and it's not stemming the flow of junk,
>why would you be running it?  Presumably you'd *do* something with the junk.


-- 
Some minds are like concrete: thoroughly mixed up and permanently set.

Bill Verity - 814-865-4758  Fax: 814-863-7049
215A Computer Building - Information Technology Services, Penn State University
At the office - on my Mac, of course ;-)