I am planning to release a new version of LISTSERV soon. One reason is to
take care of the whining from PROFS users and those unix users whose mail
program cannot parse fields of the type
From: "(Joe Doe)" <[log in to unmask]>
(admittedly ugly and definitely not intentional, but certainly not a
violation of any Internet mail header standard I know of). A more serious
reason is to package all the minor 1.7d fixes together for sites which
don't know or want to use LFIX. But the most important reason is to
introduce significant changes to LISTSERV internals and to DISTRIBUTE in
order to support the coming Netdata-capable VM mailer. This may seem like
a trivial issue, but it is more work than the new BITEARN NODES format.
You have to do something about each and every place where a line of text
is punched to the mailer. You have to do something about each and every
place where the CMS stack is used to transfer data between EXEC's when
that data may end up being sent to the mailer. The "something" is usually
not obvious, especially given the performance issues. That is the easy
part. Then you have to modify DISTRIBUTE et al to generate inter-server
jobs with longer lines, which would be trivial if you would accept to
sacrifice RSCS list processor support and to dump all pre-1.7e servers
from the backbone (after some "transition" version which would introduce
support on the receiving end while still generating on the old format).
Then you have to think about what happens to older servers when you start
sending them lines longer than they expect, and that is the difficult
part. For instance, a new CRC mechanism had to be designed, since a
server that is programmed to fold lines at 80 in order to give the mailer
a file it can handle will not get the same results as a server which
doesn't fold.
In spite of all these changes, 1.7e still won't have full support for
long mail lines. One reason is that there are areas where a lot of work
remains to be done, and where one has to worry about the impact on end
users of suddenly turning on the generation of long lines while a
significant amount of servers/mailers still only support 80 columns.
Another reason is that zilch information about the new mailer has been
released, so for instance I have no idea how to ask the mailer whether it
can handle NETDATA files and automatically enable this, and of course the
code has never been tested.
For now, 1.7e still folds messages sent to mailing lists at 80.
DISTRIBUTE and most internals can handle lines of up to 16M (in practice
limited to 64k by the network transfer formats and/or file system limits
for RECFM V files). Most other mail-based functions have a limit of
either 16M/64k or 255, in some cases I decided to still fold at 80 until
there is a majority of systems supporting long lines, even though a
higher limit is technically possible. That is, the only function which is
really limited to 80 bytes is the main mailing list code, because I have
to study the impact on older servers before I change anything, and
because the stack is used all over the place.
The remainder of this note is mostly for beta-test sites, who can start
applying now, even though I haven't yet decided whether to start beta
testing tomorrow or next week (it will depend on how much damage
LISTSERV@SEARN turns out to do to older servers during alpha testing). To
make life simpler, please sign off the beta-test list if you had beta
tested 1.7d and cannot do it with 1.7e.
The most severely impacted function is DISTRIBUTE. Netdata is now used
for inter-server jobs, so that older servers can process jobs generated
by the new servers without change. This will introduce a serious
performance degradation for 1.7c and earlier versions, I apologize to
LISTEARN sites but there was no other workable solution. The new jobs may
contain a comment card of random length in the middle, this is not a bug
but a "filler" line to align the Netdata stream on a card boundary so
that the RSCS list processor can be used. The Netdata headers on these
jobs are a minimal set - only the fields required by the standard. This,
and the fact that they are generated on the fly, makes them look weird
when PEEKed. With CMS8 (and maybe CMS7), you will be told you haven't got
enough storage to view it, and then a bunch of stacked commands will be
executed; there is an APAR for that, and a circumvention is to use a very
large virtual machine (12M, or then 30M). Assuming PEEK doesn't trip on
the bug, the reported filename will be "*............................" or
something equally unexpected. Again, this is none of my doing and
definitely not a bug in LISTSERV, DMSDDL just has a stupid default value
for an optional header field, and no I won't bother fighting this one
with IBM, especially as I don't use PEEK myself.
1.7e will mark DISTRIBUTE jobs with another CRC signature, '2-xxxxxxxx',
which older servers should ignore but forward on. 1.7e can check both old
('1-xxxxxxxx') and new CRC's, and unfortunately it will have to let jobs
with an error on the new CRC through if the old CRC matches, as older
servers are likely to fold/truncate jobs with long lines and you then
have a choice between accepting them or sending them to the postmaster,
who will either resend them (same results) or throw them away. When this
happens, a message will be printed on the console but NOT mailed to
anyone. This is not an error, it means two 1.7e servers talked to each
other through an old server which folded the lines.
BSMTP is now used for delivery of all DISTRIBUTE messages, both to
simplify the new code and for fear that the BITNET 2 exit might not fully
support Netdata and/or long address lines. Unfortunately, this will slow
down the mailer since BSMTP header processing is much slower than the old
BITNET 2. Delivery to local recipients will be faster with 1.7e due to
other changes, and the combined effect is that the mailer might build up
a queue much faster than before, so you will want to watch out for that.
Eric
|
