************************************************************************* * LMail: a simple, efficient, user and administrator friendly VM mailer * * * * Copyright Eric Thomas 1992 * ************************************************************************* Abstract -------- LMail is an RFC822 mailer for VM that was designed for simplicity, robustness, performance and, perhaps most importantly, user-friendliness and ease of configuration and maintenance by administrators with a moderate knowledge of VM and BITNET. LMail supports the NETDATA format to deliver mail with lines longer than 80 bytes, mail forwarding, a variety of "user preferences" that can be set individually by end users, and provides additional value to LISTSERV sites. LMail comes with both user and installation guides and is available under the same conditions as LISTSERV. The LMail charter ----------------- LMail was not written to advance the state of the art or to provide a new global communication tool to the user community, but as a punctual solution to a very narrow problem: 1. LISTSERV development was stalled pending the release of version 2.09 of the traditional (Crosswell/Wagner) VM mailer. 2. The traditional VM mailer was too I/O-intensive and could no longer keep up with LISTSERV. Because the bottleneck was I/O, not CPU, system commands such as SET SHARE were inoperative and manual action was required to temporarily stop LISTSERV while the mailer cleared queues of over a thousand files. LMail was chartered to provide a quick yet robust solution to these two issues, and, in addition, to avoid repeating past mistakes: a. The code should be easy to maintain. b. The software should not require the administrator to write a complex front-end in REXX to keep tables in synch with the files they are built from. Generally speaking, the software should be administrator friendly and require as little configuration files as possible to provide a "basic service". c. The software should not require obsolete input files such as XMAILER NAMES whose continued existence for compatibility introduces a number of additional issues - for instance, the multiple-mailers problem. d. The software should be user-friendly and provide additional value to end-users commensurate with today's expectations. If unix can have meaningful delivery messages, there is no reason VM can't. Now, one could spend months designing The Perfect Mailer and adding all sorts of bells and whistles to it, but the author considers mailers to be no different from screwdrivers - just a tool that can prevent you from getting some important and urgent work done but which you wouldn't want to spend your life designing and improving. 2 weeks was deemed to be the absolute maximum time to produce a working screwdriver, and it was decided to just stop after that amount of time and release whatever had been produced, provided of course that it was operational. Fortunately, progress has been surprisingly swift and the charter was actually fulfilled before the deadline, with several days left to polish up and write documentation. This was made possible through the re-use of about 15,000 lines of code from LISTSERV, packaged into a large "server development library" - an object code library rather than edited source code which would have to be maintained in parallel with the original LISTSERV code. That is, LMail will automatically benefit from any improvement made to this code for LISTSERV through a simple re-link operation. This "library" will be distributed as a large TEXT file to sites which do not run LISTSERV but need the ability to make local changes. Beyond this library, LMail contains only 180 lines of assembler code - mostly static tables which are more conveniently initialized in assembler. Knowledge of S/370 assembler is not required to update these tables, which actually contain a number of ready-to-use "slots" for locally written exits. The rest is written in the same languages as LISTSERV: PASCAL (5700 lines) and REXX (2500 lines). Overview -------- LMail provides most of the functions offered by the traditional VM mailer, with the following notable exceptions: - The 'BITNET 2' exit is not supported for outgoing mail. Only 10 sites in BITEARN NODES still make use of this exit, and a number of them are VM systems whose BITEARN NODES entries are outdated or incorrect. The only nodes actually impacted are a handful of MTS systems, for which the DIRECT exit (LMail's name for 'DEFRT 1') will be used. - LMail does not and will not implement NJE file forwarding. This has nothing to do with mail and is best implemented in a separate server. - LMail does not implement the MAILLIST command (which creates mailing lists that can then be manipulated by pre-defined list owners), or any of the exits that implemented SMTP-style mailing lists. There is LISTSERV for "serious" mailing lists, and the forwarding system provides multi-user aliases suitable for small, static lists which do not have to be changed often. LMail provides a number of additional functions: - Once the initial configuration is performed, no human intervention is needed beyond the provision of new versions of DOMAIN NAMES and BITEARN NODES. LMail will rebuild its tables automatically during startup if it detects a change, and automatically reboots every day. LMail will automatically warn the maintainer if it suspects BITEARN NODES is outdated. - LMail uses LISTSERV's INSTALL system for software updates. This means the server will collect update files until it determines it is ready to install a new version, and either do so automatically or notify you so that you can start the installation at your convenience. - LMail supports a RELEASE command and command submission via NJE files, which makes it possible to poll the servers from a remote monitor and warn system administrators of outdated configuration files. - LMail's forwarding system is hacker-proof, and can be used for userids of any length. It does not require a reboot or a configuration file change - just a SET FORWARD command. - LMail allows individual users to select a number of "user preferences" which define various aspects of mail delivery. For instance, users can turn various messages on or off, decide whether the subject line is to be included in the "New mail" message, and so on. - LMail's delivery errors are generated in a format which will allow a future release of LISTSERV to automatically take action on deleted accounts. - LMail provides added value to LISTSERV sites: a. LMail supports the 'owner-/-request' control lists introduced with release 1.7b of LISTSERV but, sadly, not used by many sites due to its lack of support (without local change) in the present version of the VM mailer. b. LMail allows RFC822-only LISTSERV lists to be created without having to open a VM account by the same name. c. LMail produces a 'Return-Path:' line which can be useful to list owners when diagnosing delivery problems. In addition, when forwarding mail for a single user, it will write the original address in the 'Received:' line (using fully legitimate RFC821 syntax) to make it easier to identify the recipient causing the original failure. d. LMail always uses the RFC822 header, not the MAIL FROM: field, to determine what to show in the "New mail" message. The VM mailer produced different results depending on the exit being used. - LMail provides limited online help and lets users order copies of the documentation via TELL commands. Compatibility ------------- LMail is not compatible with the Crosswell/Wagner mailer - it is a totally different implementation of the various network standards. In fact, LMail is more similar to LISTSERV than to the VM mailer. But from the point of view of end users, they provide roughly the same service. In particular, LMail does support incoming files without BSMTP headers, as many servers use this format to communicate with their mailer. Performance ----------- Good performance was one of the main goals in the development of LMail, but not one where much time has been spent. The poor performance of the VM mailer is not due to bad data structures or algorithms, but to inefficient use of system interfaces, especially in the BSMTP code (as you will see below). It was clear from the beginning that no special effort would be required to achieve considerably better performance with LMail. There is room for improvement in a number of areas - the charter simply dictated to go for the shortest possible development time, and the result which was achieved (about a factor of two reduction in CPU time and a factor of 5 in I/O) seems fully satisfactory. Here are a few benchmarks. VCPU and TCPU are shown first, followed by I/O count and elapsed time. The I/O figures come from VM/ESA's CP IND USER, and should be taken with a grain of salt given that they include both disk and terminal I/O. In addition, it is unclear how spool I/O is accounted for depending on the use of SIO vs diag X'14'. A much better measurement of I/O savings is to look at the elapsed time - it is I/O, not CPU contention, which makes the mailer wait. Note that the accuracy of the elapsed time is 1 second only. MAILER is V2.08, and, except for test number 6, all incoming files are in BSMTP format. <1> Local delivery of 20 small files. MAILER: 1.12/3.19 I/O=322 Elapsed: 23 sec LMail: 0.50/1.04 I/O= 60 Elapsed: 4 sec <2> Same as 1, but the files are forwarded via BSMTP to another mailer. MAILER: 1.06/3.15 I/O=322 Elapsed: 22 sec LMail: 0.58/1.43 I/O= 80 Elapsed: 5 sec <3> One file to forward to 20 recipients via BSMTP. MAILER: 0.36/1.22 I/O= 70 Elapsed: 5 sec LMail: 0.15/0.55 I/O= 26 Elapsed: 2 sec <4> One 10k records file, locally delivered. MAILER: 15.69/32.79 I/O= 27 Elapsed: 50 sec LMail: 0.67/ 2.64 I/O= 4 Elapsed: 7 sec <6> Same as test 2, but the files are submitted without BSMTP headers. MAILER: 0.41/2.00 I/O= 62 Elapsed: 8 sec LMail: 0.49/1.27 I/O= 60 Elapsed: 5 sec Test <2> - test <6> MAILER: 0.65/1.15 I/O=260 Elapsed: 14 sec LMail: 0.09/0.16 I/O= 20 Elapsed: 0 sec Note that the extra LMail I/O count is due to an additional message per file. For LMail, the difference is less than 0.01 TCPU per file, whereas with MAILER the BSMTP exit generates an enormous amount of extra I/O and costs another 0.7 second of wall-clock time per message. For more information -------------------- Subscribe to LMAIL-L@SEARN and make sure to INDEX the archives after you join.