A new database, called LISTS, has been developed for backbone release 1.5n
sites which can afford the disk space. It stores extensive information about
all the registered lists in the LISTSERV network. That is, it contains
information about the same lists that are in the LIST GLOBAL output, but the
complete list header is available and can be searched using the standard
LISTSERV database functions.
Technicalities:
- Only backbone sites with CMS release 4 or 5 can host the LISTS database. The
corresponding database driver will not work under CMS release 3.
- Sites willing to host the LISTS database should send me mail BEFORE 1.5n is
released. I will then change their backbone tags to read ":backbone.YES
LISTDB(YES)". The registration can of course be made after 1.5n is out, but
this is likely to create a lot of REFRESH requests which we should try to
avoid.
- I have strictly no idea how much disk space will be required, so don't ask
me :-) However, the files will be kept on the 'FILEDISK' disk, so you should
certainly modify LOCAL SYSVARS to set 'FILEDISK' to something different from
'A5' if you plan to host this database.
Operation:
LSVLDELT has been modified to create a checksum of the contents of the list
header (edited as per a REVIEW command - notably, all .IK/.IT calls are fully
resolved). This checksum is appended to the list title as it is stored in
LOCLIST FILE and propagated into the network for other peers to store in their
GLOBLIST FILE. Thus if the list header is changed, the list will appear as
changed even though the title et al are unchanged. Finally, the list header
checksums are also checksummed into the global 'CKS' checksum, so that a
header update lost in a spool crash will be detected with just the global
checksum (is that enough checksums for you? :-) ). LSVLDELT will send the
headers of any list flagged as modified along with the 'REP' statement.
On the input side, LSVLUPD will work as before if the LISTS database is not to
be made locally available. Otherwise it will do a lot of things to store the
headers in separate files on the FILEDISK, checking several things in the
process, and preparing a 'list of changes' for the LISTS database driver.
The problem is that this database is likely to change relatively often and to
contain quite a lot of entries. Furthermore, each entry (1 entry = 1 list) is
going to take a bit of CPU to be analyzed and indexed. It would therefore not
be a good idea to refresh the complete database index every time a change is
being made. Being provided with a list of changes, the database driver can
regenerate (or delete) the index entries of just those lists that were
updated, and keep the other ones 'as is'.
Finally, the LSVLDELT command syntax has been extended:
LSVLDELT REFRESHME <HDR>
LSVLDELT REFRESH userid@node <HDR>
LSVLDELT INIT userid@node <HDR>
The (non-default) 'HDR' option indicates that the list headers are also
wanted. 'LSVLDELT REFRESH' always causes headers to be sent out, since the
update job is forwarded to the nearest backbone server for distribution to all
the servers.
End-user's point of view:
A request to access the LISTS database will fail with a 'Database LISTS does
not exist' message if sent to a pre-1.5n server.
When sent to a 1.5n server which is not on the backbone, you will be provided
with the userid@node of the backbone server nearest to the one you sent your
request to.
A backbone server which doesn't host the database will give you the address of
the nearest backbone server that does have it.
The database's characteristics are as follows:
- DATE corresponds to the date the last update was GENERATED by the remote
server hosting the list (remote server's local time). Its resolution is the
day, ie 'yy/mm/dd' is available but time is '??:??:??'.
- LIST-ID (also LISTID or ID) corresponds to the network-wide "List-ID"
keyword contents.
- LISTNAME (or NAME) is the actual name of the list (ie VM userid thereof).
- NODEID (or NODE) is the nodeid of the server hosting the list.
- TITLE is the list title.
All the keywords defined in the various list headers are also available,
PROVIDED THAT THEY ARE NOT DEFAULTED. That is, you can do "Select * in LISTS
where send=public and owner contains ERIC@FRECP11", provided that the list
header contains an explicit definition for "Send" and "Owner". If these
keywords do not appear in the list header, the default values (resp "PUBLIC"
and <list of postmasters>) are NOT substituted. This would require too much
CPU time, and in some cases the default value can be determined only by the
server hosting the list.
The following document portions are available:
- ALL: the full list header.
- Header or HEADER: synonyms for ALL.
- ABStract or SUMmary: all the lines in the header that do not contain keyword
definitions. These are supposed to be a description of what the list is
about. In the worst case, it will contain the list title.
This SUMMARY thing is only temporary, I think we should discuss a way to
better identify 'sections' in the list headers.
Finally, the default index looks like this:
*-----------------------------------------------------------------------*
> Select * in LISTS where subscription=open and title contains mail
--> Database LISTS, 3 hits.
> Index
Ref# Listname Nodename List title
---- -------- -------- ----------
0027 MAIL-L DEARN Mail Discussion List
0028 MAILBOOK DEARN MAIL/MAILBOOK subscription list
0054 XMAILER DEARN The Crosswell Mailer List
*-----------------------------------------------------------------------*
Eric
PS: I'll be off from FRECP11 next week.
|
