Description of the changes for release 1.8b of LISTSERV(TM)
-----------------------------------------------------------
Copyright 1995 L-Soft international, Inc.
May 8th, 1995
*************************************************************************
************************** List owner's notes ***************************
*************************************************************************
The release notes for version 1.8b of LISTSERV(TM) have been split into
three documents: executive notes, list owner's notes, and LISTSERV
maintainer's notes. The present list owner's notes describe all the
changes that list owners need to be aware of, although in some cases list
owners may be impacted by changes described in the LISTSERV maintainer's
section. For instance, the availability of a new systemwide option may
prompt the LISTSERV maintainer to make a change affecting all the lists
on his machine. Thus, list owners are advised to read the maintainer's
notes as well.
Note to non-VM customers: the present release notes describe all the
changes from the date of release of version 1.8a (7 Dec 93). Because the
first VMS(TM) and unix(R) versions of LISTSERV were released in June 94,
the improvements made during the first half of 1994 were already included
in the code released as "1.8a" for non-VM systems. Similarly, due to the
release date of the Windows NT(TM) version, very few of the changes in
these release notes will be new to Windows NT(TM) customers.
**************
* Highlights *
**************
- Comprehensive, 95-page list owner's manual now available online.
- (VM) Numerous changes to facilitate Internet migration. BITNET
addresses are only used when there is no other alternative, or when
explicitly requested.
- New SCAN command and automatic delivery error monitoring system
simplify processing of delivery errors.
- "Spam" detector shields LISTSERV lists from unsolicited advertisements.
- New message templates for better configurability, including support for
"infobot" style lists, automatic acknowledgement of xxx-request
messages, top and bottom message banners for disclaimers or short
SIGNOFF instructions, and so on.
- New QUERY format with short description of individual options; new
QUERY ... WITH command to identify all subscribers with a particular
option activated.
- New subscription options, support for multiple moderators, new message
approval procedures, poster authentication options for secure lists...
******************************************
* Documentation: New list owner's manual *
******************************************
A comprehensive, 95-page list owner's manual is now available online. A
plain text version is included with the 1.8b update, replacing the old
"INFO KEYWORDS" and "INFO OWNERS" documents. You can also FTP a copy of
the new manual from FTP.LSOFT.COM, CD DOCUMENTS. The file is available in
a variety of word processing formats - check the FTP server for more
information.
****************************************************************
* Usability: Changes to default options for Internet migration *
****************************************************************
In order to facilitate the transition from BITNET to the Internet, a
number of changes have been made to default keyword and system
configuration variables in version 1.8b. Some of these changes were made
during the development of LISTSERV-TCP/IP (1-2Q94) and were thus present
in the first VMS/unix builds released in June 94. The rest were made in
3-4Q94. All changes are present in the January builds for VMS/unix, and
in the Windows builds. In other words, if you were running the latest
1.8a builds for VMS/unix/Windows, these changes are actually not new.
They are new to VM sites, and some may be new to VMS/unix sites running
an older build. Many of the changes have little relevance to non-VM sites
and are marked "(VM)".
Changes to list header keywords
-------------------------------
- (VM) The "Files=" keyword now defaults to "Files= No", reflecting the
fact that the distribution of NJE files to a LISTSERV list is now a
historical feature.
- (VM) The "List-Address=" keyword now defaults to the long list name and
Internet address (formerly, it defaulted to the 8-character list name
and the BITNET address). Non-VM servers are not affected because they
have no BITNET address and thus always use their Internet address, and
because they can support lists with longer names.
- The "Newsgroups=" keyword now defaults to "Newsgroups= None", causing
LISTSERV not to generate any "Newsgroups:" field. This tag was required
by old list<->news gatewaying software and is now considered obsolete,
whereas the presence of a "Newsgroups:" field in mail headers is
misleading subscribers into thinking the list is gatewayed to usenet.
The "Newsgroups=" keyword is still supported of course, and lists which
have an explicit "Newsgroups= a.b.c" in the list header will continue
to generate a "Newsgroups:" tag.
- The "X-Tags=" keyword now defaults to "X-Tags= Comments" rather than
"X-Tags= Yes". While this change is not specifically related to the
Internet migration, it was scheduled to be made at the first
opportunity.
- The TOCOUNT/NOTOCOUNT suboption of the "Loopcheck=" keyword is no
longer supported. This ancient loop checking mechanism is now totally
obsolete and has not been carried over when the loop detector was
rewritten for portability. This option was already turned off by
default, and in practice the only difference is that list headers
containing this option will have to be edited the next time they are
stored.
Changes to server configuration defaults (set by maintainer)
------------------------------------------------------------
- (VM) The MAXGET and MAXGETK variables were raised from 50 messages
totalling 2 megabytes to 200 messages totalling 16 megabytes. These
variables are used by the default LSV$GETQ exit shipped with the VM
version of LISTSERV and the change may or may not have any effect
depending on local customization. The temporary file server functions
currently supported by the non-VM versions do not use the LSV$GETQ exit
and are thus not affected by this change.
- The default value for the PRIMETIME variable was changed to the empty
set, that is, LISTSERV now processes "overnight" jobs immediately. This
change reflects the fact that, with today's networking technology,
holding jobs until the night shift is no longer appropriate except in a
few specific cases, and thus should not be the default. List owners
should note that most sites made this change manually over the last 5
years or so, ie in most cases the policy is already in effect.
- The default values for FILEMAXL and MAILMAXL have been increased to
50,000 and 10,000 lines, respectively. Mail sent to the LISTSERV
address (as opposed to mail sent to a list) is no longer check against
the MAILMAXL threshold.
Other miscellaneous changes
---------------------------
- Header default: the default header format was changed from SHORT to
FULL headers, for the convenience of MIME users. Note that some mail
software packages still have maximum header size limits dating back to
the early 80s. Because FULL headers are much larger, it is possible
that some of your subscribers will complain about not receiving their
mail any longer. In that case, you may want to try switching them to
SHORT headers to see if the problem disappears. L-Soft recommends
configuring mailers to accept at least 50 header lines. L-Soft's
products accept up to 100 kilobytes of header data, and the only
purpose of this restriction is to reduce the impact of large
misformatted mail messages (such as misdirected VM batch console logs,
which can be millions of lines long) on the system.
- (non-VM) Null record padding removed: because the VM file system does
not support null (zero-length) records, mail software on VM systems has
traditionally been unable to generate zero-length lines. While LISTSERV
does internally support such null lines even on VM, it is not possible
to pass them on to the VM SMTP software. Similarly, VM LISTSERVs will
always receive one-blank records from the VM SMTP server, because the
(non L-Soft) SMTP software stages the data through an unformatted disk
file. Thus, the VM version of LISTSERV is currently restricted in its
ability to properly receive and generate null lines. To guarantee the
compatibility of server-to-server requests, the original non-VM
versions had code to convert all null records to one-blank records. The
concern was not that the new VMS/unix code would not be able to handle
null records properly, but that existing, older versions of the VM
LISTSERV would not be able to process such requests, in which case the
customers in question should at least be warned of the impending
incompatible change. Fortunately, extensive testing has revealed that
both versions 1.7f and 1.8a are able to process null-record requests
from non-VM servers, and the null record padding code has simply been
removed from the non-VM versions.
***************************************
* Usability: New subscription options *
***************************************
Three new subscription options have been added to facilitate the
management of large lists. These options can be set by the list owner
only, using the SET command.
The EDITOR option allows a subscriber to post messages to a moderated
list, without having to go through the moderator. It is virtually
identical to adding the subscriber's address to the "Editor=" keyword,
but easier to manage. The only difference between the EDITOR option and
the "Editor=" keyword, other than not being visible in the list header,
is that the "Editor=" keyword also defines a (seldom used) access level
class which can then be used in keywords such as "Review=". Thus, one
could have a list with "Review= Editor", indicating that only the users
listed in the "Editor=" keyword are allowed to review the list. The
EDITOR option does not confer this privilege. Note that the EDITOR option
is only meaningful on moderated lists.
The REVIEW option indicates that all postings from the subscriber in
question should go to the list owner for approval. By default, the
messages are forwarded to the primary list owner (the first address in
the "Owner=" list). An alternate moderator (or list of moderators) can be
designated using the "Moderator=" keyword. Note that adding a
"Moderator=" keyword will not make the list a moderated one; it is the
"Send=" keyword that controls whether the list is moderated or not. The
"Moderator=" and "Editor=" keywords simply define moderator/editor
addresses to be used when and if moderation is called for, such as when a
user with the REVIEW option posts a message or when "Send= Editor" is in
effect. For lists discussing sensitive topics, it might be appropriate to
set "Default-Options= REVIEW" so that all postings from new subscribers
go through the moderator during a "probation period". Once the
subscribers have shown that they are able to participate in the
discussion in a productive way, the REVIEW option can be selectively
removed by the list owner.
The NOPOST option can be used to prevent a particular subscriber from
posting to the list. LISTSERV returns the postings to the user with the
following message: "You are not authorized to post to the XYZ-L list. For
more information, please contact the list owners at
[log in to unmask]". Note that this option is only effective on
private lists. With "Send= Public", the user could still post from a
different account; with "Subscription= Open", the user could leave the
list and re-subscribe.
The antonyms of EDITOR, REVIEW and NOPOST are NOEDITOR, NOREVIEW and
POST, respectively.
Because these options are stored in the subscriber's entry in the list,
and not in the list header, they may not be effective with peered lists
unless the user is subscribed to all the peers. For instance, a
subscriber with the EDITOR option will only be granted editor privileges
if he submits his posting to the peer on which he is subscribed (which in
practice should not be a problem). For NOPOST and REVIEW, it may be
necessary to add the user to the other peers and set these extra
subscriptions to NOMAIL.
IMPORTANT: In order to add these new options, the format of columns
81-100 of the list file returned by the GET command had to be changed.
LISTSERV will continue to accept the old format as input and will not
reformat old entries until they are modified (to minimize user impact in
case the LISTSERV maintainer should need to fall back to an old version).
However, you will no longer be able to locate (say) subscribers with the
CONCEAL option by looking for a 'C' in column 92. Use the new QUERY ...
WITH CONCEAL command for this purpose.
*******************************************************
* Usability: Change in MAIL, DIGEST and INDEX options *
*******************************************************
Originally, the MAIL, DIGEST, INDEX and NOMAIL options corresponded to
the four possible settings of a single option. That is, switching from
DIGEST to NOMAIL would stop the delivery of new digests, as expected, but
would also make LISTSERV forget that the subscription should be in digest
mode. A subsequent switch to MAIL would restore delivery, but in normal
(non-digest) mode. This was confusing to novice users.
In version 1.8b, the MAIL/NOMAIL option has been isolated from
DIGEST/INDEX. The MAIL/NOMAIL option controls whether messages should be
delivered, and the DIGEST/INDEX/NODIGEST/NOINDEX option controls the
format in which messages should be delivered. Thus, switching to NOMAIL
and back to MAIL now preserves the digest/index/normal delivery setting.
To provide as much compatibility with the old syntax as possible, the
four options now operate as follows:
- DIGEST: enable digest delivery mode (which negates INDEX), enable mail
delivery. No change from version 1.8a.
- INDEX: enable index delivery mode (which negates DIGEST), enable mail
delivery. No change from version 1.8a.
- NOMAIL: disable mail delivery. No change from version 1.8a.
- MAIL: restore mail delivery, without altering the digest/index/normal
delivery setting (new behaviour). For compatibility with 1.8a, if mail
delivery was already active, the MAIL option negates INDEX/DIGEST.
Thus, a user going from NOMAIL to MAIL will keep his previous delivery
options, whereas a user going from DIGEST or INDEX to MAIL will in fact
deactivate index/digest mode.
To revert from digest/index subscription mode to normal delivery, you can
use either the MAIL option as before, or the NODIGEST/NOINDEX option. The
NODIGEST and NOINDEX options were actually present in versions 1.7f and
1.8a, as synonyms for the MAIL option. In other words, you can update
your instructions to indicate that the DIGEST/INDEX options are negated
by the NODIGEST/NOINDEX options, even if your server is not yet running
version 1.8b.
*******************************
* Usability: New SCAN command *
*******************************
A new command has been introduced to make it easier to locate a
subscriber's entry in the list given the subscriber's name or approximate
address. The syntax of the new command is:
SCAN listname string1 <string2 <...>>
This will search the address and name fields of all the subscribers in
the list for matching entries. Case is ignored unless you enclose the
search string in double quotes. For instance, 'SCAN XYZ-L Joe' will match
'JOE', 'Joe' and 'joe'; 'SCAN XYZ-L "Joe"' will only match 'Joe'. If
multiple search strings are specified, the entry must contain ALL of them
in order to be selected. For instance, if you receive a delivery error
indicating that one "Smith, Joe" has left XYZ Corporation and that his
account has been deactivated, you can locate all the Joe Smiths in your
list with 'SCAN XYZ-L Joe Smith'. This will find 'Joe Smith', 'Smith,
Joe', 'Joe H. Smith', etc.
A limited version of the SCAN command is available to non-privileged
users that are authorized to review the list. The syntax is the same, but
concealed subscribers are skipped during the search. That is, concealed
entries are totally ignored during the search (as opposed to being
searched, but without being displayed at the end). Since concealed
entries do not participate in the search, it is impossible for the user
to gain any information about concealed participants through the SCAN
command.
Note that you can, of course, continue to use the QUERY command to make
searches on the address field. For instance, 'QUERY XYZ-L FOR *smith*@*'
will display all the subscribers whose username contains the string
'smith' in either case. The QUERY command, however, cannot search the
name field, and returns a lot of unnecessary information. The SCAN
command only shows you what you are really interested in: the user's name
and e-mail address.
*************************************
* Usability: Enhanced QUERY command *
*************************************
The QUERY command has been enhanced to make it easier to identify users
with a particular subscription option. For instance, you may wish to
check for NOMAIL users regularly, or for users with the CONCEAL option.
To do this, simply issue the following command:
QUERY listname WITH option1 <option2 <...>> FOR pattern
For instance, QUERY XYZ-L WITH CONCEAL FOR *@* will return a list of all
the subscribers which have the CONCEAL option. This new WITH option is
only available to list owners.
*********************************************************
* Usability: Enhanced reporting of subscription options *
*********************************************************
The QUERY command has been enhanced to provide a more meaningful
description of the various available subscription options. Previously,
the subscription options were reported as follows:
----------------------------------------------------------------------
Subscription options for Eric Thomas <[log in to unmask]>, list TEST-L:
Ack=No, Mail=Yes, Files=Yes, Repro=No, Header=Full, Conceal=No
----------------------------------------------------------------------
While compact, this format was not very helpful to inexperienced users.
Here is an example of the new format:
----------------------------------------------------------------------
Subscription options for Eric Thomas <[log in to unmask]>, list TEST-L:
MAIL You are sent individual postings as they are received
FULLHDR Full (normal) mail headers (formerly "FULLBSMTP")
NOREPRO You do not receive a copy of your own postings
NOACK No acknowledgement of successfully processed postings
----------------------------------------------------------------------
Options which are considered "important" are shown regardless of their
setting. This makes the user aware of the existence of the option, and
the possibility to change it if the current setting is not satisfactory.
The more exotic options are listed only if they are in effect, to avoid
confusing the user with a long list of available options.
**************************************************
* Usability: Automatic delivery error monitoring *
**************************************************
In order to give list owners more flexibility in meeting their particular
policies for mail "bounces" while allowing the tedious delivery error
processing tasks to be delegated to LISTSERV, delivery error monitoring
and reporting functions have been added in version 1.8b. Whereas version
1.8a would immediately remove subscribers from the list upon the receipt
of a delivery error indicating a permanent error ("No such local user",
"No such host", etc), the new error monitor will keep track of the kind
and amount of errors received, and take action based on a number of
configurable parameters. The new monitor can also be run in "manual"
mode, in which case it merely generates daily condensed reports for the
list owner, without automatically removing users from the list.
Naturally, the old 1.8a behaviour remains available, although it is no
longer the default option.
By default, version 1.8b will now wait 4 days before removing a user from
the list as the result of permanent delivery errors. This value strikes a
balance between the desire to tolerate human or computer error over a
"long weekend", and the need to keep the number if delivery errors
processed by LISTSERV to a reasonable value. To avoid overwhelming the
LISTSERV host with large numbers of delivery errors for active lists, the
monitor will also delete the user upon the receipt of 100 permanent
delivery errors (this value can be increased). Finally, the monitor will
notify the user of the deletion, so that, if for any reason the problem
had actually been solved, the user will know what happened.
The delivery error monitor is configured through the "Auto-Delete="
keyword. The basic syntax remains unchanged:
Auto-Delete= No
or:
Auto-Delete= Yes,option1,option2,...
The SEMI-AUTO and FULL-AUTO options remain unchanged from version 1.8a.
Here is a list of the new options for the delivery error monitor:
- The MANUAL option configures the monitor in passive mode. Every day,
you receive a report with a list of bouncing addresses, statistics for
each individual address, and a copy of the last error message (one-line
abstract). It is then up to you to decide how to handle these errors.
- The DELAY(nn) option lets you change the "grace period" during which
the monitor will tolerate permanent errors without taking any action.
DELAY(0) means that the users should be deleted immediately upon the
receipt of a permanent delivery error (1.8a behaviour). The default
value is DELAY(4), which should be adequate for most purposes. A
shorter value might be appropriate for very active lists. A larger
value is not recommended, and in particular values close to 7 should be
avoided, because most "false bounces" occur over the weekend. In other
words, a value of 7 does not provide much additional tolerance compared
to a value of 5, because it is during the weekends that errors are
likely to occur. If 5 is not lenient enough, the next logical choice
would have to be 10 or 11.
- The MAX(nnn) option defines the maximum number of errors that the
monitor will tolerate for any given user. The default is 100. After
receiving this many errors, the monitor will delete the user regardless
of the DELAY(nn) option. While this value can be increased, you should
check with your LISTSERV maintainer before doing so. The threshold is
unlikely to be exceeded unless your list is very active, in which case
the number of bouncing addresses is likely to be in the 20-100 range.
100 bouncing addresses times 100 tolerated bounces per address is
alreay 10,000 messages for the LISTSERV host to process. Depending on
the hardware configuration, this may or may not be an issue.
In order to understand how the monitor works, it is important to realize
that it only receives notifications of NON delivery. The monitor has no
way to know that the problem has been corrected, other than to note that
no error has been received in a certain time frame. Generally speaking,
the monitor counts errors, keeps track of their severity, and remembers
when they occurred. When certain criteria are met (as explained above),
the monitor deletes the user. After a certain period of time without new
errors, the monitor assumes that the problem has been solved and stops
monitoring the user in question. The following tips may prove helpful:
- Remember to increase the MAX threshold if you make a large increase to
the DELAY threshold. Otherwise the users may be deleted anyway.
- When decreasing the DELAY threshold, expect a large number of deletions
the first day. This is because the monitor probably has a number of
users on record that have been bouncing for the amount of days that you
specified, but for less than the old (larger) threshold. It does not
necessarily mean that the monitor will continue to delete users at that
rate. Because most errors occur over the weekend, it takes about a week
to get an idea of how many users will be deleted with a particular
threshold value.
- Making a big increase to the DELAY threshold to provide more leniency
during a holiday may not be a good idea. While it will indeed disable
the monitor for the duration of the holiday, switching back to the
normal threshold when you return will cause the monitor to delete all
the users that had been bouncing during the holidays. In general, you
should avoid making temporary changes to the DELAY threshold, because
it takes the monitor a while to adapt to the new settings.
- The best way to relax the rules during a long holiday is to leave the
DELAY threshold unchanged but switch the monitor to passive mode
("Auto-Delete= Yes,Manual"). Noone will be deleted over the holidays,
but the monitor's cycle will not be perturbed. When you return, you
should wait about a week before switching back to automatic mode. This
is because, after a long holiday such as Christmas, it usually takes
about 2 working days for system administrators to solve all problems.
In some cases, the problems will have caused bounces to remain
undelivered. So, by fixing the problems, the system administrators may
actually send a flood of new bounces corresponding to problems that
have now been solved. Unfortunately, since the monitor only receives
NON-delivery reports, it has no way to know that these problems have in
fact been solved. As a rule of thumb, you will note that your daily
delivery error reports are much longer than usual over the vacation.
When you return, you should wait until they are back to their normal
size before switching back to automatic mode.
IMPORTANT: Regardless of the way you instruct LISTSERV to handle them,
delivery error reports indicate that a message has been lost. One cannot
overstress the fact that false permanent errors ("No such user" when the
user exists, "No such host" when the host does exist) are a VERY SERIOUS
PROBLEM which should be addressed by the corresponding system
administrators, and not simply ignored. Make sure to tell your
subscribers that this problem affects ALL messages destined to them, and
not just the messages from your list. Because their mail system was
rejecting any and all mail to their mailbox over a certain period of
time, they may have lost very important private mail along with the
messages from your list.
******************************************************************
* Usability: New import procedure for non-LISTSERV mailing lists *
******************************************************************
A new list "import" procedure has been added in version 1.8b to
facilitate the migration from non-LISTSERV mailing list managers. For
best portability, this procedure has been implemented as a LISTSERV
command option rather than a system script. This also makes it easier for
list owners to use the import facility on their own, for instance when
merging two lists.
The first step in migrating a non-LISTSERV list to LISTSERV is to prepare
a list header with the new list. L-Soft intends to develop a GUI
interface for this purpose, but currently this must be done with a
regular text editor. Because of the large number of non-LISTSERV list
managers and the greater flexibility of LISTSERV, and after analysing
feedback from customers who had successfully migrated to LISTSERV, it was
felt that an automatic conversion procedure for the list's configuration
options would not be very useful. Indeed, most of the sites that migrated
to LISTSERV immediately took advantage of the new configuration options,
and found it simpler to prepare a new configuration based on their
business needs than to attempt to convert the configuration information
from their old list manager.
Once the list is created (with an empty subscriber list), the subscriber
data from the old list manager can be merged in using the new ADD IMPORT
command. The first step is to prepare a text file containing the
following information:
ADD listname DD=SUB IMPORT
//SUB DD *
...
/*
where '...' is replaced with the subscriber data from the old list
manager, in its native format, with one subscriber address and/or name
per line. You do not need to indicate the old list manager's brand as
LISTSERV supports most major formats and will recognize them
automatically.
Experienced list owners will have noted that this is the same as a
regular ADD job, except for the IMPORT option. This option tells LISTSERV
to suppress all notifications (as per a QUIET ADD), to use the new
"import" parser when analysing the various addresses, to suppress all
successful responses (only errors are reported), and to speed up
processing by considering the whole job as a single transaction, whereas
in a regular ADD job there is one transaction per subscriber. This
"transaction" difference relates to the way LISTSERV manipulates the list
data and to the implications in case a hardware or software failure
should occur during the execution of the job. With a normal ADD job,
users are notified as each entry is processed. To guarantee that the
users have actually been added when they receive their notification, and
to prevent duplicate notifications from being issued in case a failure
should occur and the job should have to be retried, the subscriber
entries are checkpointed as they are added to the list. If the job fails
after processing 500 entries, the 500 entries that were added to the list
at that point are guaranteed to be correct. With the IMPORT option, the
transaction works in an "all or nothing" mode, and has to be restarted
from scratch in the event of a hardware or software failure.
NOTE: Due to the multitude of possible syntaxes and options to check, an
import job for a large list (over 1,000 subscribers) can take several
minutes to complete, depending on the hardware being used and on whether
you are using the High Performance or regular version of LISTSERV. The
server prints progress messages to its log at regular intervals to
confirm that it has not gone into a loop.
*****************************************
* Usability: Automatic "spam" detection *
*****************************************
Ever since the media started talking about the "Information
Super-Highway", the Internet has drawn a lot of attention from the "man
in the street". What a few years ago was mostly an academic and research
network is turning into a general purpose resource; some think that in
another few years, every household will be connected. This is a profound
cultural change and, like most cultural changes, it comes with its share
of good things and bad things. One of the bad things is that some
unscrupulous people have realized that the charging model used by the
Internet service providers makes it possible to use other people's
resources and money to deliver advertisements to millions of people, for
a one time cost of a few dollars. Taking advantage of the chronical lack
of legislation in the networking area, they are actively distributing
"spams" - unsolicited advertisements sent to large numbers of mailing
lists, with no consideration for whether or not the product being offered
is relevant to the topic of the list. Thus, we have seen advertisements
for thigh creams posted to all the known mailing lists, causing an
outburst of traffic for which many people have to pay by the byte. Each
of these spams is estimated to cost over $100,000 to the victims,
collectively, in the form of higher connection charges. This is real
money that actually shows up on people's bills and has to be paid. In
addition, the spams and the resulting flood of complaints to the service
providers (often directed to the lists by mistake) waste hundreds of
thousands of dollars in manpower, machine time, people's time wasted due
to slow/unavailable service, etc. These spams have to be stopped.
The root of the problem is that, most of the time, it is the recipient
who pays for the message, not the sender, because to a large extent this
is where the costs are incurred, and there is no mechanism (to date) for
the Internet service providers to charge back other providers' customers
in the way that phone operators do. Imagine what your mailbox would look
like if any company could send you junk mail at YOUR expense! However,
there is no consensus that such a change in accounting principles would
be generally beneficial; for instance, if people had to pay for thousands
of deliveries in order to answer a question on a large mailing list,
there would be many questions and very few answers, making the mailing
list totally useless. At any rate, this is not a change which can be done
overnight. Similarly, "anti-spam" legislation is at best a long term
prospect, whereas a solution is needed immediately. There are already
books with simple, step-by-step instructions for preparing a spam and
reaching as many people as possible, and they are selling very well.
Fortunately, LISTSERV's nature as a distributed network of interconnected
servers makes it possible to identify and eliminate these spams before
they do much harm. While it is virtually impossible for a small isolated
server to detect a spam (unless the sender listed the thousands of lists
he was targeting in the "To:" field), for the simple reason that it will
only ever receive a few copies for its own public lists, the LISTSERV
network as a whole receives thousands of copies of the spam. By comparing
notes with each other, the servers can quickly determine that a spam is
occurring and raise a network-wide "spamming alert", stopping the message
before it does much damage at all.
When LISTSERV determines that a spam is occurring, it blocks all further
copies of the message and forwards them to the list owner for manual
verification. The poster is informed once, by the server that first
raises the alert (due to the distributed nature of LISTSERV, the poster
may actually be informed more than once if several servers identify the
spam at the same time). The poster is NOT informed when subsequent
messages are rejected. That is, if the poster is the innocent victim of a
computer error, he will know that something went wrong. If, however, he
is indeed a spammer, the poster will not know which lists were missed and
may have to be retried. If fact, spammers seldom read the replies they
receive from LISTSERV servers, simply because there are over 6,600 public
lists, and most of them will return a positive or negative
acknowledgement. The message is sent from a "junk account" whose mail is
not read, or then filters are used to automatically discard unwanted
computer replies.
As always with automatic procedures, there is a risk of false alerts.
Ultimately, what determines whether a message is a "spam" or
"cross-posted material" is its relevance to the topic of the list.
Unfortunately, LISTSERV is not capable of determining whether a
particular message is relevant to a particular list. All it can do is
note that many copies of a particular message have been sent in a short
time frame, and assume that the message is a spam. To take a concrete
example, a conference announcement, complete with registration form and
conference fees, could be a highly relevant item or a spam depending on
which lists it is posted to. The poster may just be trying to reach as
many people as possible in the hope of increasing attendance (this has
actually happened with conferences on multiple occasions), or he could be
posting the announcement to germane lists, with the list owner's prior
approval. There is simply no way for a computer to tell. When a genuine
message needs to be posted to large numbers of mailing lists, L-Soft
recommends sending the message to the list owners, rather than posting it
directly (this is already required for closed lists anyway). The list
owner can then determine whether the message is relevant, and forward it
to the list as appropriate. Another advantage of this method is that the
subscribers know for a fact that the owner authorized the use of the list
for this purpose, and will not accuse the poster of having abused their
list.
List owners who wish to disable spam detection for their lists can do so
by adding:
Loopcheck= NoSpam
to the list header. When spam detection is disabled, postings are always
accepted.
Conversely, the spam detector does not guarantee that spams will never
make it to the lists. In particular, the LISTSERVs cannot tell that a
message is a spam until they have received a certain number of copies. In
other words, when it receives the first copy of the spam, the LISTSERV
network has no way to know that 5,999 other copies are on their way, and
the message will be processed normally. Due to the distributed nature of
LISTSERV, with all the servers working in parallel on the
"investigation", dozens of messages will actually be treated as "first
copies" and distributed to the lists. The bulk of the messages, however,
will be stopped, and this is what really matters.
Until legislation is in place to provide a more satifactory solution, we
are likely to see a typical "armour and shield" race. For this reason,
L-Soft will not disclose any information about the methods use by the
spam detector, and will keep improving the algorithms constantly. If the
algorithms were disclosed, it would take about 3 to 6 months for a new
edition of the "Make money fast" books to hit the bookstores, and we
would all be back to square one. Luckily, the odds are in the list users'
favour, because the spammers are not really interested in a technological
race. Their only goal is to reach millions of people quickly and cheaply
and, by definition, they have absolutely no interest in the nature of the
audience that they are reaching; the only thing that matters to them is
numbers. Thus, a LISTSERV subscriber is worth as much to them as a
Majordomo subscriber or a usenet reader, and there is no reason for the
spammers to try to break LISTSERV's defences at all costs when usenet
remains totally undefended.
************************************
* Usability: New message templates *
************************************
Version 1.8b introduces many new configurable message templates, and, in
particular, two new types of message templates for "linear" and optional
messages. Traditionally, message templates have contained the text of
"long" administrative messages, such as messages informing subscribers
that they have been removed from a mailing list. These notices were sent
unconditionally, as a separate message. The template processor now
supports "linear" messages, which are sent as a normal command reply and
allow the list owner to modify the replies from selected commands, and
"optional" messages, which are only sent if a template for this action
has been specifically provided by the list owner. Here is a list of the
new template messages:
- SUB_CLOSED (linear): this is the message that is sent to a subscriber
attempting to join a list with "Subscription= Closed". The default is
"Sorry, the &LISTNAME list is closed. Contact the list owner (&OWNER)
for more information."
- SUB_OWNER (linear): this message is sent to a subscriber attempting to
join a list with "Subscription= By owner". The default is "Your request
to join the
&LISTNAME list has been forwarded to the list owner for approval. If
you have any question about the list, you can reach the list owner at
&OWNER." Because this is a linear template (see below), it is not the
best place to put long questionnaires, application forms, terms and
conditions, or other material that the subscriber should be required to
review prior to joining the list. See the "Tips" section below.
- POST_EDITOR (linear): this is the message LISTSERV sends to people
attempting to post to the list, if it is moderated. The default is
"Your &MESSAGE has been submitted to the moderator of the &LISTNAME
list: &MBX(&MODERATOR)."
- TOP_BANNER, BOTTOM_BANNER (optional): when these templates are present,
their contents are automatically inserted at the top (respectively
bottom) of each and every message posted to the list. Typically, the
top banner would be used for a copyright or short legal warning which
absolutely has to be seen by each and every reader. The bottom banner
could contain instructions for signing off the list, a disclaimer, an
acknowledgement of a sponsor's contribution, a "tip of the week", etc.
- REQACK1: this message is sent automatically in reply to any message
sent to the xxx-request address. The message acknowledges receipt,
explains the difference between the LISTSERV and xxx-request addresses,
and contains instructions for joining and leaving the list. To suppress
this message for your list, simply redefine it in the
'listname.MAILTPL' and use the .QQ instruction:
>>> REQACK1 This message is not wanted for our list
.QQ
- AUTODEL1: this is the message that is sent to users who are deleted by
the delivery error monitor. You can customize it to fit your needs, or
suppress it using the same procedure as for REQACK1.
- POSTACK1 (optional): when present, this message is sent in reply to any
message posted to the list. This is very useful for creating
"infobots", or just for returning a standard acknowledgement to
contributors. The &SUBJECT variable contains the subject of the
original message, and naturally the usual substitutions (&LISTNAME,
&DATE, &TIME) are available.
- ADDREQ1 (changed): this message, which was already present in version
1.8a, is sent to the list owner when a user requests to join a list
with "Subscription= By owner". In version 1.8a, a copy of the message
was sent to the subscriber, to confirm that the request had indeed been
forwarded to the list owner. Unfortunately this was confusing to the
many novice users who do not understand the difference between primary
and secondary message recipients ('To:' vs 'cc:'). In version 1.8b,
only the list owner is sent a copy of the ADDREQ1 template. If you were
using this template to send new subscribers a questionnaire,
application form or similar material, you will need to add a '.TO
&WHOM' instruction to your modified template, as by default the user
will no longer receive a copy.
In a linear message, most special instructions are ignored. This is
because the contents of the template are just a few lines out of a larger
message that is being prepared by LISTSERV to contain the reply to the
user's command(s). For instance, you do not have any control over the
"Reply-To:" field of the message, because the message in question is
shared with other commands and, in fact, may not be a mail message at all
but an interactive message to the user's terminal, a GUI request, etc.
Generally speaking, with a linear message you are providing the TEXT of
the reply to be shown to the user, but you do not have any control over
the methods used for delivering this information.
Tips:
- Many list owners require prospective subscribers to fill in a little
questionnaire before being added to the list, or to explicitly state
that they have read the list charter and agree to follow all rules or
be removed from the list. The most convenient method, for both list
owner and subscriber, is to have the SUBSCRIBE command return a copy of
the questionnaire (or list charter, etc), and not forward the request
to the owner. The user answers the questions and returns them directly
to the list owner, who then adds the subscriber manually. Naturally, it
is more convenient for the user if this information arrives in a
separate message, with a 'Reply-To:' field pointing to the list owner's
address. Thus, you should not use the SUB_OWNER template for this
purpose, because it is a linear template and does not give you any
control over the 'Reply-To:' field. The SUB_OWNER template could be
modified to read "A copy of the list charter is being sent to you,
please read it carefully and follow the instructions to confirm you
acceptance of our terms and conditions." The list charter would then be
sent separately, through the ADDREQ1 template. You would use the .RE
OWNERS command to instruct LISTSERV to point the 'Reply-To:' field to
the list owners, and .TO &WHOM to change the destination from list
owner to subscriber. If you want to receive a copy of the message, you
can use .TO &WHOM cc: [log in to unmask]
- When writing templates, it is a good idea to use substitutions (&XXXX)
for information which may change in the future. In particular, it is
not uncommon for lists to have to be moved from one host to another,
and this will be a lot easier if the template uses substitutions for
the list address and list host. The &LISTADDR substitution translates
the full address of the list ([log in to unmask]), whereas &LISTNAME is just
the name (XYZ-L). For references to the server and host, use &MYHOST
for the Internet hostname, &MYSELF for the server address (normally
LISTSERV@&MYHOST), and &OWNER for the xxx-request mailbox address.
These substitutions are "universal" and can be used in all templates.
For instance, if you decide to make a bottom banner with instructions
for leaving the list, the text could read: "To leave the list, send a
SIGNOFF &LISTNAME command to &MYSELF or, if you experience
difficulties, write to
&OWNER."
**********************************************
* Usability: Enhancements to list moderation *
**********************************************
Support for multiple moderators
-------------------------------
Traditionally, moderated LISTSERV lists have allowed multiple "editors"
(ie multiple authorized contributors who may submit messages without
going through the moderator), but all contributions from regular members
were sent to a single address, often called the "first" or "primary"
editor because of the convention that the first person listed in the
"Editor=" keyword was the moderator, and that the second and following
ones were editors. While the moderator's address could of course be a
mail alias pointing to several people, there was no simple mechanism for
coordinating the work of multiple moderators. The moderators had to
inform each other of which messages they had processed and which they had
left for the other moderators to process. Usually, teams of moderators
would adopt a simple convention, deciding for instance that messages from
people whose name starts with A-L go to a particular moderator. However,
each had to read all the submission, and this does not scale up to large
teams of moderators.
With version 1.8b, you can make LISTSERV assign messages to multiple
moderators automatically through the use of the new "Moderator=" keyword.
Simply list the addresses of all the moderators in the group, and
LISTSERV will send each of them a fair share of the traffic (ie if you
have 4 moderators, each of them will get 1/4th of the messages). If you
want a particular moderator to receive more than a fair share of the
messages, simply repeat his addresses until the desired proportion is
reached. Each address in the list corresponds to a fair share. Thus, if
you have 3 moderators (Joe, Jean and Julie) and you want to send half of
the traffic to Joe, use "Moderator= Joe,Joe,Jean,Julie".
Addresses listed in the "Moderator=" keyword are automatically editors.
Thus, if you have a simple setup with just one moderator/editor, you do
not need to provide an "Editor=" keyword. For compatibility with older
versions, if no "Moderator=" keyword is present, LISTSERV uses the first
address from the "Editor=" keyword.
New message approval method
---------------------------
By default, when LISTSERV receives a message from a regular subscriber to
a moderated list, it simply forwards it to the moderator. On some lists,
where the moderator actually edits the messages (for instance by creating
a manual digest), this is entirely satisfactory. On other lists, the
moderator merely screens incoming messages for compliance with the
charter, relevance to the topic, etc. In that case, the moderator
essentially acts as a simple accept/reject filter. To accept the message,
the moderator forwards it back to the list, and LISTSERV distributes it.
A special mail header convention allows the moderator to tell LISTSERV
that this is a forwarded copy of an accepted message, and LISTSERV
updates the mail headers accordingly, making the message appear to come
from the original poster, and indicating which of the moderators approved
the message. Unfortunately, with some mail packages it can be very
difficult to obtain the desired combination of mail headers. While it has
the advantage of allowing minor updates "on the fly", this procedure may
not be the most convenient for all audiences.
In version 1.8b, an alternate procedure can be activated by changing the
"Send=" keyword from "Send= Editor" to "Send= Editor,Hold". The "Hold"
option instructs LISTSERV to keep the messages on hold after they have
been forwarded to the moderator. The moderator can then approve the
message through LISTSERV's "OK" procedure, ie by replying to the message
and typing just "OK". Naturally, the moderator is free to make changes to
the message, but in that case the modified copy will need to be posted to
the list since LISTSERV does not have it on file. In the interest of
expediency, the moderator need not do anything to discard a message;
LISTSERV will quietly and automatically delete unapproved messages after
a week. Note again that the messages are also forwarded to the moderator,
and thus expiring the messages merely removes LISTSERV's copy. The
expiration delay can be increased (but not decreased) using the
"Confirm-Delay=" keyword. Make sure to inform the LISTSERV maintainer
before any significant increase.
*********************************************************
* Security: New authentication option for list postings *
*********************************************************
While LISTSERV offers a number of security mechanisms (configured through
the "Validate=" keyword) to protect mailing lists, these mechanisms had
so far applied only to the execution of LISTSERV commands, and not to
list postings. That is, while one could prevent hackers from making
unauthorized changes to the list configuration or membership, there was
unfortunately no mechanism to protect the material being posted to the
list. Because Internet mail can be forged by non-privileged users,
malicious users could submit postings to a list in other people's name
and, in particular bypass "Send=" access control by making their message
looks like it came from someone who is authorized to post to the list.
With version 1.8b, it is now possible to use of the "OK" mechanism to
confirm the authenticity of list postings, by modifying the "Send="
keyword to read "Send= ...,Confirm" (where the ellipsis represents the
previous value of the "Send=" keyword). When this option is activated,
LISTSERV puts all incoming messages on hold and sends a confirmation
request to the original poster. The message is not processed until the
confirmation is received.
Because this procedure is tedious and potentially confusing to
non-technical users, it should only be activated on lists which need this
level of security. For instance, send confirmation would not be
appropriate on a large open public list. It is ideal, however, for
announcement/newsletter lists and other low-volume lists with a well
defined number of authorized contributors.
*******************************************************
* Usability: New option to facilitate list relocation *
*******************************************************
When relocating a LISTSERV list from one system to another, you can now
instruct the LISTSERV running the old list to forward all commands and
postings to the new address, and return an explanatory message to the
sender with instructions for using the new address. This is done through
the "New-List=" keyword. Simply add "New-List= [log in to unmask]" to the old
list header, and store the list. Once you insert the "New-List=" keyword,
all commands except PUT will be forwarded to the new host. LISTSERV will
also stop advertising the old list (as if you had added "Confidential=
Yes") so that all subscription requests are sent to the new host.
Finally, you will be required to strip the list of most of its
configuration keywords before you can store it back with a "New-List="
keyword, to reflect the fact that the list no longer exists and that
these keywords have no effect.
*************************************************
* Usability/Security: Changes to the PW command *
*************************************************
The syntax of the PW command for general users has been simplified in
version 1.8b. In addition, the authentication procedures have been
strengthened through the use of the "OK" mechanism. In version 1.8a, a
customer-provided exit was responsible for filtering and authenticating
PW commands. In most cases, the exit either did no authentication at all,
rejected all requests not issued through a secure channel (such as CP
SMSG on VM or LCMD on VMS/unix), or simply forwarded them to a human
being for verification. While the exit remains available in 1.8b (and
does not need to be refitted), LISTSERV now authenticates commands
received from insecure channels through the "OK" mechanism, in addition
to any authentication that may be done by the exit. This higher degree of
security makes it possible to eliminate the LSV$PW exit in most cases.
The new syntax of the PW command is as follows:
- 'PW ADD password' will register the specified password, as before.
- 'PW CHANGE newpw <PW=oldpw>' can be used to change your password. As
before, you have the option of validating the change with your old
password. In addition, you may also omit the old password and go
through the "OK" procedure.
- 'PW RESET' will, after authentication through the "OK" procedure, reset
your password and allow you to use the PW ADD command to select a new
password. You no longer need to rely on the LISTSERV maintainer to
assist you when you forget your password.
- The PW DELETE command, while still supported, is now obsolete.
IMPORTANT: LISTSERV passwords only provide "weak" authentication, because
the users transmit the password in clear text over the Internet. Once the
password has been compromised, it can be reused at will. The "OK"
mechanism provides a stronger degree of authentication because it cannot
be reused. That is, while a malicious user installing a "sniffer" on an
ethernet would be able to intercept the information necessary to confirm
an "OK" request on behalf of an innocent third party, this would only be
possible for as long as the sniffer is active. With passwords, on the
other hand, the knowledge gained while the sniffer is in place can be
reused multiple times.
********************************
* Miscellaneous: Minor changes *
********************************
This section contains a list of minor changes which, while noteworthy, do
not warrant a long description.
- (VM) Long list names now used consistently: when a "long" list name has
been defined through the "List-ID=" keyword and the "List-Address="
keyword is set (or defaults) to the "long" ID, LISTSERV will now use
the long name consistently in all messages. Previously, LISTSERV
accepted the long name in user commands, but always used the short name
when replying.
- New LIST GLOBAL format: the format of the LIST GLOBAL response had to
be changed to accomodate the much longer Internet addresses. Each list
is now split across two lines, one with the name and address of the
list, and one with its description. Internet addresses are also used
for LISTSERV-NJE servers whenever possible.
- Generalized use of Internet addresses: generally speaking, LISTSERV now
uses Internet addresses in messages wherever possible. BITNET addresses
are only used when there is no alternative.
- New "Notebook-Header=" keyword: this new keyword lets you select the
amount of header information you want LISTSERV to store in the list
archives. The default, "Notebook-Header= Short", is compatible with
version 1.8a and corresponds to the 'SHORT' header set. Setting
"Notebook-Header= Full" will use the 'FULL' header set for the
archives. Note that enabling this option can double the amount of disk
space used up by your archives!
- (VMS/unix) The "Renewal=" keyword is now supported in the non-VM
versions.
- Administrative messages shortened to 73 columns: because many PC mail
interfaces display less than 80 columns of text, administrative
messages have been reformatted to fit in 73 columns.
- Changes in header option names: to reflect the fact that the so-called
"BSMTP" headers have been the default header type for all users since
version 1.8a (and for Internet users since version 1.6a), whereas the
so-called "RFC822" headers are only retained for compatibility with
historical software implementations, the header option names have been
renamed as follows: the old SHORTHDR and FULLHDR options were renamed
to SHORT822 and FULL822, respectively, and the SHORTBSMTP and FULLBSMTP
options were renamed to SHORTHDR and FULLHDR. The xxxBSMTP options are
still accepted and produce the desired result; however, users who still
absolutely need the "RFC822" headers will have to use the new
SHORT822/FULL822 option names. Note that "RFC822" vs "BSMTP" headers
actually refer to the name of delivery "exits" for the so-called
"Crosswell Mailer", the first implementation of a RFC822 mail transfer
agent for VM. Both sets of headers are RFC822 compliant, but the
"BSMTP" headers could only be submitted to the Crosswell Mailer's
through its "BSMTP interface", which was not available in the first few
versions. Again, this distinction is purely historical and totally
irrelevant to Internet users. For all practical purposes, the "BSMTP"
headers are the normal RFC822 headers and the "RFC822" headers are an
old type of header required by very old versions of the Crosswell
Mailer.
-------------------------------------------------------------------------
L-SOFT and LISTSERV are trademarks of L-Soft international.
Unix is a registered trademark of UNIX Systems Laboratories, Inc.
VMS is a trademark of Digital Equipment Corporation.
Windows and Windows NT are trademarks of Microsoft corporation.
All other trademarks, both marked and not marked, are the property of
their respective owners.
-------------------------------------------------------------------------
|
