[. . .]
>OK, the list is going to be fairly big in the future and we have to be sure
>the auto-delete process is functioning properly. I have been sending about
>20 mails to the TESTLIS with the above mentioned configuration. But none of
>te bougus mail-addresses has been deleted..... This is strange... Help is
>needed...
The short answer is that in order for LISTSERV to process a bounce
automatically the bounce must be in a format that LISTSERV
understands. There are two or three formats that LISTSERV
understands (most notably that described in RFC 1894) but many
mailers don't use those formats. If the bogus addresses you are
using to test the auto-delete function are all from the same domain
then it's likely that the mailer responsible for handling mail for
that domain is such a mailer.
I'll include more information on the subject of how LISTSERV handles
bounces at the end of this message.
>A second issue is if it is advisable and/or possible to set the FROM
>address-space with the lists own e-mail address?!?!
http://www.lsoft.com/manuals/lsv-faq.stm#5.14 addresses this issue.
LISTSERV won't make changes to the From: address, so the answer is
basically no. The usual strategy is to set up a generic email
address ([log in to unmask] or whatever) and use that as the return
address in your email client when you send the mailing.
Information on how LISTSERV handles bounces
-------------------------------------------
One of the problems that everyone who runs a mailing list has to deal
with is one of bad addresses; if your mailing list has a subscriber
whose email address is invalid then every message that goes out results
in a nondelivery notification (sometimes referred to as an 'NDN' or
'bounce') being generated, which for a high-traffic mailing list can
result in a very large number of bounces.
These bounces have to be dealt with somehow, and generally the
responsibility for doing so falls on the person or people running the
mailing list. Fortunately, LISTSERV has some tools which allow it to
take some of the burden off of the listowner; unfortunately, none of
these tools can be 100% effective.
After reading this document you should have a better understanding of
how LISTSERV handles nondelivery notifications and some of the issues
involved. There are some things that are glossed over and simplified,
however, so you may want to also look at some of the resources referred
to in the final section of this document to get a fuller perspective.
A Simple SMTP Transaction, and RFC821's "MAIL FROM:" vs. RFC822's "From:"
-------------------------------------------------------------------------
Before discussing what happens when a message is undeliverable, though,
let's look at what happens when a message is delivered successfully.
There are several Internet standards dealing with getting email from one
place to another. One of the most important is RFC 821, titled "Simple
Mail Transfer Protocol" (or SMTP for short). This lays out how one mail
server connects to another, what commands can be used, and what
responses are allowed and what they mean.
Generally the mail server that wants to deliver the message connects to
port 25 of the receiving mail server and starts sending commands. (If
you are not sure exactly what this means that's fine.) Here is a sample
conversation between two mail servers; items preceded by '>>>' are being
sent by the mail server that's sending the message, and those preceded
by '<<<' are being sent by the receiving server.
<<< 220 home.ease.lsoft.com (LSMTP for Windows NT v1.1b) ESMTP server ready
>>> HELO earthlink.net
<<< 250 home.ease.lsoft.com says hello to earthlink.net
>>> MAIL FROM:<[log in to unmask]>
<<< 250 Ok
>>> RCPT TO:<[log in to unmask]>
<<< 250 Ok
>>> DATA
<<< 354 Start mail input; end with <CRLF>.<CRLF>
>>> From: [log in to unmask]
>>> To: [log in to unmask]
>>> Date: Fri, 18 Feb 2000 17:34:21 -0500
>>>
>>> Thanks a lot!
>>> .
<<< 250 Ok, message queued (73 bytes)
>>> QUIT
<<< 221 home.ease.lsoft.com says goodbye
Here's what happened above. The receiving SMTP server detected a
connection attempt and announced itself ready to receive commands. The
sending SMTP server identified itself as earthlink.net and the receiving
SMTP server acknowledged that.
Next the sending SMTP server issued the MAIL command (indicating it has
mail to send) followed by the FROM:<return-path-address>. This
return-address is extremely important because it instructs the receiving
SMTP server where to send back any NDNs that are generated as a result
of the transaction. Note that this may or may not be the same as the
From: address that appears in the headers. (More on this later.)
Next the sending SMTP server says who the message is being sent to (in
the RCPT TO: line). If there were more than one recipient each
recipient would get its own RCPT TO: line, but in this case there's only
one.
Finally the DATA command is issued. This tells the receiving SMTP
server that the following lines are the actual message that's being
sent. First any header lines the sending SMTP server wants to specify
are sent, then a blank line, then the body of the message ("Thanks a
lot!"). Finally a line consisting of a single period is sent, and that
tells the receiving SMTP server that the DATA section is over. The
receiving SMTP server reports back that the message was accepted, and
the two servers say goodbye to each other and close the connection.
Some things to notice:
In this example the MAIL FROM: address was the same as the "From:"
address given in the DATA section, and the RCPT TO: address was the same
as in the To:.
Neither of these identities has to be the case in general. For
instance, if you send a message and "Bcc:" someone, they will definately
get their own "RCPT TO:" line, but they will not appear in the message
headers at all.
Similarly, when a message is distributed by a LISTSERV-hosted mailing
list, the From: address will be the address of the person who originally
sent the message, but the MAIL FROM: will be something like
"[log in to unmask]" (or wherever your mailing list is
hosted). This makes it so that subscribers to the mailing list can see
who sent the message, but any bounces that are generated by the mailing
go to the owner- address rather than the original sender of the message
(who probably would not be in a position to do anything about them).
A metaphor may be of use here. When someone sends you a physical piece
of mail, it comes in an envelope which includes the address of the
person who sent the message and your address. When you receive the
message you open and discard the envelope and read the letter itself.
The letter may also contain information on who the message is from and
who it is to, and this information may differ from the information that
was on the envelope. If the letter can't be delivered for some reason
it is stamped 'Return To Sender' (or similar) and returned to the
sender's address on the envelope.
Internet mail standards are based on a similar idea. The 'MAIL FROM:'
and 'RCPT TO:' addresses are equivelent to the addresses on the envelope
(and in fact you may sometimes see them referred to collectively as the
'SMTP envelope' or similar). The actual message headers and the body of
the message are equivelent to the letter itself, and they should not be
referred to at all when the message is being delivered (though sometimes
broken mail servers do make reference to them).
Throughout this document I will make a distinction between the address
used in the MAIL FROM: and the one used in the From: of the same
message. As the latter is specified in RFC 822, "Standard For The
Format Of ARPA Internet Text Messages", I will sometimes refer to the
"RFC 822 'From:' address" in distinction to the "RFC 821 'MAIL FROM:'
address".
LISTSERV's Automatic Bounce Detection
-------------------------------------
As previously noted, LISTSERV always sends out mailing list messages
with a MAIL FROM: pointing to 'owner-listname@[your site]'. Any bounces
generated by a mailing should thus go to the owner-listname address.
LISTSERV monitors the owner-listname address and any mail that is
delivered there is assumed by LISTSERV to be a bounce. If the Auto-Del=
setting is set to Full-Auto or Semi-Auto LISTSERV then attempts to
understand the message. If it can understand the message it then acts on
it, and if not it forwards it to the listowner.
This works reasonably well, but there are a few pitfalls.
1) Bounces are not always sent to the correct address. According to
Internet standards any bounces generated are supposed to go to the MAIL
FROM: address, but there are mail servers out there that do not fully
comply with the standards. The result is that you can expect to see
some bounces go to virtually any email address in the headers of the
message: The "From:", the "Sender:", the "Reply-To:", etc.
One result of this is that occasionally bounces are sent to the mailing
list itself. Fortunately LISTSERV has ways to detect when this is
happening so that the bounces are not distributed to the mailing list;
suspected bounces are instead forwarded to the owner.
A more pernicious problem is that some bounces will go to addresses that
have nothing to do with LISTSERV. Since LISTSERV can't process messages
it never sees, this means that such bounces will have to be dealt with
manually. Unfortunately since this is the result of mailers you have no
control over misbehaving there isn't much that can be done about this
problem apart from recognizing that it exists.
2) LISTSERV recognizes bounces that are in some standard formats.
(Specifically it recognizes bounces in the format described in RFC 1894
and a couple of other formats.) Unfortunately, there are many
non-standard formats around which LISTSERV cannot recognize--some
nondelivery notifications don't even say what email address resulted in
the bounce. Since LISTSERV doesn't understand these messages it
forwards them to the listowner. (This is true even if you have
Autodelete=Yes,Full-Auto set.)
3) Someone may sign up for your mailing list with one address and then
set things up so that address forwards to a second address. (For
instance someone may sign up as [log in to unmask] and then have that
forward to [log in to unmask]) If something then goes wrong with the
second address LISTSERV will be notified that that address is bad, but
it won't be able to do anything about it as that address isn't signed
up, even if the bounce is in a recognizable format.
Depending on your Auto-Delete= settings LISTSERV will either forward
such bounces to the listowner or will just add the address to the daily
monitoring report. (Auto-Delete=Yes,Semi-Auto does the former,
Auto-Delete=Yes,Full-Auto the latter.)
4) There are several different types of bounces. There are 'temporary'
bounces which indicate that if you try sending your message again later
it may get through, and 'permanent' bounces which indicate that you
should not retry sending the message. LISTSERV gets the information on
whether a bounce is temporary or permanent from return codes which are
defined in RFC 821 and RFC 1893. So for instance a "550" bounce is a
'permanent' bounce that generally indicates that the recipient email
address is unknown (e.g. it doesn't exist).
The problem is that the codes may be used incorrectly. For instance,
some sites report "Mailbox full"-type errors as 550 errors even though
the problem will be cleared up as soon as the user cleans out his or her
mail account. (Most sites will use the correct code for this, which is
552.)
The key point is that as far as the standards are concerned (and so as
far as LISTSERV is concerned as well) the numerical code is the only
thing that matters, regardless of what any qualifying text may. Sites
using the codes incorrectly may see their users incorrectly getting
removed from mailing lists because of this. All the listowner can do is
add the affected subscribers back and inform them of what the problem is
that caused them to be deleted from the mailing list.
Passive Probing
---------------
One of the banes of large, active mailing list is the large volume of
bounces which LISTSERV gets but can't interpret and so which must be
handled manually by the listowner. Starting with LISTSERV 1.8d passive
probing was introduced which allows LISTSERV to handle most bounces,
even if they are not in a standard format. Here's how it works:
Let's say that you have Auto-Delete=Yes,[. . .],Probe(30). When a
mailing goes out, 1/30th of the subscribers are chosen randomly to be
probed. Those subscribers are sent seperate messages, each of which has
a customized MAIL FROM:. Then LISTSERV knows that any mail sent to one
of the customized MAIL FROM:s is almost certainly a bounce related to
the corresponding email address's subscription. It can then act on that
knowledge even if the actual bounce is not in a format that it can
understand. Here's a more concrete example:
Let's say that the mailing list address is [log in to unmask] and one of
the subscribers is [log in to unmask] The MAIL FROM:
address would be set to something like
owner-listname*badaddress**HOME*-EASE*-LSOFT*[log in to unmask]
Now suppose that the subscribed bad address is bad. A bounce is
generated and is sent to the MAIL FROM: address, in this case
owner-listname*badaddress**HOME*-EASE*-LSOFT*[log in to unmask] LISTSERV
sees that a message was sent to that address and extracts the name of
the mailing list (listname in this case) and the offending address,
which is [log in to unmask] It then deals with it
according to the Auto-Delete= settings.
This is a very powerful approach and using it results in a very
substantial decrease in the amount of time the listowner has to spend
dealing with bounces. (Active probing uses essentially the same
mechanism to detect bounces.) There are some disadvantages, however:
1) Misbehaving mail servers that don't send to the correct address will
continue to be a problem.
2) LISTSERV can now process bounces that aren't in a format it can
understand, but when it does so it's essentially flying blind--any
message sent to the MAIL FROM: address will be treated as a bounce.
This includes autoresponses, 'Mailbox Full'-type messages, and
nondelivery notifications warning of a temporary delivery problem.
3) The MAIL FROM: addresses that are used to probe are completely legal
in that they fully comply with the relevent standards. However they are
not the type of addresses that one runs into much on a day-to-day basis,
and some mail servers may reject mail with such a MAIL FROM:. If this
appears to be happening it is most commonly because the mail server has
been configured to reject mail from addresses containing asterisks
('*'s), though it could also potentially be because of the length of the
address. Often in this case if you explain the situation to the email
administrator and convince him or her that you are not trying to do
anything wrong they will modify the rules to fix this, but of course
there is no guarantee.
Where to Go for More Information
--------------------------------
Both the List Owner's Manual and the Site Manager's Manual contain a lot
of information on autodeletion and active and passive probing which is
not discussed here. The manuals are available in a variety of formats
from http://www.lsoft.com/info/default.asp?item=manuals
Internet standards relating to SMTP can be found at
http://www.imc.org/rfcs.html#hosttohost and those relating to the format
of email messages, including headers, can be found at
http://www.imc.org/rfcs.html#rfc822 . For the purposes of this document
the most important standards are:
RFC 821, "Simple Mail Transfer Protocol"
( available at http://www.imc.org/rfc821 )
RFC 822. "Standard For The Format Of ARPA Internet Text Messages"
( available at http://www.imc.org/rfc822 )
RFC 1123, "Requirements for Internet Hosts -- Application and Support"
(in particular chapter 5, "Electronic Mail -- SMTP and RFC-822")
( available at http://www.imc.org/rfc1123 )
RFC 1893, "Enhanced Mail System Status Codes"
( available at http://www.imc.org/rfc1893 )
RFC 1894, "An Extensible Message Format for Delivery Status Notifications"
( available at http://www.imc.org/rfc1894 )
--
Jacob Haller, Technical Support
L-Soft international, Inc
http://www.lsoft.com/
|
