From the 1.8c Maintainer's Release Notes: ***************************************************************** * Performance: Enabling reverse indexing for database functions * ***************************************************************** (VM) Note: the guidelines in this section only apply to the new database functions. The original VM database functions are not currently affected by the DBRINDEX configuration variable. The new 1.8c database functions can be configured to work in one of two modes: with forward indexing only, and with both forward and reverse indexing. The forward index is the file called xxx.DBINDEX. With list archive files and other plain-text based databases, this file tells the LISTSERV database functions where a particular database entry begins and ends. Thus, entry #17283 in the XYZ-L list might begin at line 281 of XYZ-L.LOG9609 and extend until line 312. The forward index also contains frequently accessed information, such as the subject of a message or the date at which it was posted. However, it does not contain any information that would help in locating all the entries that contain the word 'TEST'. The reverse index (xxx.DBRINDEX), when enabled by setting the configuration variable DBRINDEX to 1 (or letting it default to this value), provides this functionality. A reverse index will dramatically speed up searches on large databases. However, it will not have much effect on smaller databases. Without reverse index, you can still expect a search rate on the order of 1-3M per second (elapsed) on a typical PC server. Many lists have archives in the 1-5M range, and would not really benefit from reverse indexing. The drawbacks of reverse indexing are: 1. The reverse index file uses up (typically) a few megabytes of disk space, even if the archives are relatively small. This is because, even with a small list, tens of thousands of different words are likely to be in use. This could be an issue for sites with thousands of small lists. 2. Building and maintaining the reverse index uses up some amount of CPU time. This could be a problem on time-sharing systems where CPU cycles are billed by the hour, and I/O accesses are comparatively cheap. 3. The current implementation of the reverse indexing code may require significant amounts of virtual memory for large databases. On a system with virtual memory quotas, this would require increasing the quotas for the LISTSERV process. On a dedicated system, it could, in some extreme cases, require a hardware upgrade. None of these problems are likely to be an issue with a typical dedicated configuration. However, we have customers running over 6,000 lists on the same machine, paying an outsourcing company for every hour of CPU time used, or running LISTSERV on machines with 8M of RAM, and this is the context in which we are mentioning these issues. An operating system specific discussion follows: - NT (default = 1): L-Soft recommends leaving reverse indexing enabled unless your system has less than 32-64M of RAM (depending on RISC/CISC architecture and on the size of your largest archives) or is already paging. Most dedicated PC servers have enough resources to enable this option without impacting overall system performance, although in some cases a RAM upgrade could be necessary. - unix (default = 1): L-Soft recommends leaving reverse indexing enabled unless your system has less than 32-64M of RAM (depending on RISC/CISC architecture, presence or absence of X-Windows and size of your largest archive) or is already swapping heavily. Most dedicated unix servers have enough storage to enable this option without impacting overall system performance. - VM (default = 0): L-Soft recommends enabling reverse indexing on VM/XA and VM/ESA unless all your lists have small archives. On S/370 systems, only about 10-12M of virtual storage can be made available to LISTSERV, which is sometimes insufficient even without reverse indexing. This option can only be enabled safely on an XA/ESA system, after switching the LISTSERV userid to an ESA/XA or XC machine and increasing its VMSIZE to the recommended value of 64M (sites which already need more than 32M due to the number of lists they are hosting should add another 32M). While this may seem excessive, LISTSERV is unlikely to actually use all that storage. Database accesses last a few seconds at most, and any peak in storage usage would be temporary. - VMS (default = 0): L-Soft recommends enabling reverse indexing on systems which have 64M of memory or more (possibly less for VAX systems or systems without DECwindows), and which are not page-bound. On the other hand, you will need to increase the PGFLQUO quota for the LISTSERV process before enabling this option. Depending on the number of lists on your system and on the size of the archives for your largest list, you will want to increase PGFLQUO by 16-64M. VAX users should note that the default PGFLQUO set by the 1.8b installation procedure was found to be much too conservative (regardless of the reverse indexing issue) and has been doubled for version 1.8c. Reverse indexing is disabled by default on VMS as it is likely to lead to virtual storage exhaustion unless the quotas are increased. - Windows 95 (default = 1): since Windows 95 uses the same executable as Windows NT, reverse indexing is enabled by default. However, a typical Windows 95 system with 8-16M of RAM cannot support reverse indexing and this option should be disabled. When installing with the graphical installation program, your SITE.CFG file will be automatically modified to disable reverse indexing. If you install the new version manually, you will need to make this change yourself.