I have just finished to process the replies to the LSTSRV-L survey about
LISTSERV. Here are the general trends:
- Everybody is willing to accept a few ascending compatibility problems if
there is no other way to blend the LISTSERV and NETSERV command sets (only 1
negative reply). 30% of the people who replied said that they were getting
confused with the two command sets because they are similar and yet
different enough to make it easy to send an invalid command -- for instance,
"AFD ADD password fn ft" versus "AFD ADD fn ft PW=password".
- Nobody feels that LISTSERV eats too much CPU (one reply was "a lot", three
replies were "little", the rest was "normal").
- Everybody is willing to spend some virtual storage or some DASD space to
reduce CPU, even when they admitted that CPU was not a concern at all at
their installation. I also got some funny replies like "ok, but I won't give
it more than 12M -- that's my last word". To clarify my point, I was
speaking of an increase in virtual storage of at most 1M or 2M, that is a
recommended storage size of 3-4M. As for DASD space I meant a few hundred
kbytes to hold "summary" files (see the new PEERS NAMESUM file). Less than 2
cylinders anyway.
- The same holds for DASD I/O, to a lesser extent.
- Most sites can give CPU cycles to help the network but not DASD space. A few
sites can give DASD but not CPU, and some sites can give really nothing.
- Everybody thinks DISTRIBUTE is underused and is willing to participate in an
ARPA backbone, provided it does not represent too much local load on the
server. A few sites added "provided it is used for academic distribution,
not games" and I treated that as a "NO" reply as it is out of question to
start red-penning ARPA distribution lists. If the stuff has to be
distributed, it will be sent anyway and my personal opinion is that the best
possible method should be used, even if it means my LISTSERV will be used to
distribute junk mail about the last DALLAS episode. I can understand that
other sites may have a different opinion, but it is not possible technically
to "sort out" lists and say "ok, use my server for INFO-AMIGA INFO-VAX but
not for SF-LOVERS ATARI-LOVERS " or suchlike.
- What are the slowest commands?
o MAIL processing when the input file is 3,000 records and there are 150
recipients. Now how do you want me to solve that? :-) I'd say you ought
to use DISTRIBUTE in that case (not just to reduce your CPU load).
o ADD/SUBSCRIBE when the list is large (100+ users).
o Storing a list. Actually, any form of the PUT command is slow, but
storing lists is the slowest.
o GET or just any command that makes reference to a file when you have 30
filelists and some of them are 1200 records large...
The rest was said to be acceptable -- at least considering the work it's
doing. You can't expect a full NODESGEN to take 15 seconds :-)
I have made an analysis of the above commands to determine why they are so
slow. At present I have only examined the MAIL/ADD/SUBSCRIBE commands, I'll
have a look at the file server stuff next week. MAIL is slow because EXECIO is
slow and because RFC822 takes time to parse. There is not much I can do about
it. However, the other commands are slow because they manipulate the huge
SIGNUP FILE (1500 records on my server but probably much more at hub nodes).
Besides, this file eats up a large amount of DASD space because it is recfm F
(so that it can be updated). I have therefore decided, based on the survey
input, to spend some virtual storage and a little CPU time at initialization
to save DASD space and a lot of CPU when processing commands.
- The SIGNUP file is now stripped and kept in RECFM V format -- 50% savings.
- New entries are consequently always appended at the bottom, even if the
entry already exists.
- Its contents are read into GLOBALV storage at initialization, and the file
is compressed if required. This takes some time but it's done only when the
server is rebooted. You might therefore want to have your LISTSERV rebooted
once a day at midnight or something like that.
You would be surprised at how much faster ADD/SUBSCRIBE and PUT (list) are
on a large list with this little mod. It costs around 50kbytes of storage
(with 1500 lines of SIGNUP FILE -- should be a linear increase) but I think
it's really worth it. It saves you around 40kbytes of DASD space too (still
for 1500 lines).
I will be looking into using some GLOBALV storage for the file server
functions, but it's much more complicated: if you have 30 filelists of up to
1200 entries, you'll need an awful lot of storage so it's not possible to just
keep everything in storage. I could 'cache' the latest 'nnn' entries, though,
but I'm not sure it would be a big help. I'll see.
Eric
|