On Thu, 23 Feb 1995 19:59:02 EST Roger Fajman <[log in to unmask]> said: >Well, the incident that prompted my question was a new list owner >attempting to add 10,000 subscribers to his new list in 6 humongous >files of multiple ADD commands. It bogged down LISTSERV on our poor >little 9370-60 for hours and we still ended up deleting most of the >files. The ADDs were QUIET, so the notifications weren't the cause. The algorithms for lists of that size were significantly enhanced in FIX18AX2. The time to fire up the ADD commands does cost something, but it would be neligible compared to the inherent slowness of adding users to a large list without the improved algorithms (in fact there was nothing wrong/broken with the old algorithms, they just didn't have optimizations to linearize performance for large lists). Now that it takes 0.2 sec wall clock on the average to add a user to a list with 100k subscribers on a PC, the overhead for firing up ADD 10k times might make a difference, and I have no idea how much. I always use the DD form when adding a bunch of users because it's simpler once you know the syntax and I know it will be somewhat faster. I've started a 10k ADD job on my PC in the background and it's clearly going to take several minutes, although I suspect a large fraction of the elapsed time is spent scrolling the messages in the DOS box. The PC definitely doesn't feel loaded. Maybe if I could type HT it would go faster. I guess I should just wait until the list has grown to reasonable size. All right, it has reached 7000 now and the PC feels slower while the DOS box is not scrolling quite as fast as before. I think it's no longer limited by the scrolling speed. It seems to be doing about 10 a second now. >Eric mentioned a fix that we don't have that's supposed to speed up ADD, >so that might help some, but it's unclear to me if it would be enough. >Eric was vague (in a private exchange) about whether the ADD command job >you describe would be significantly more efficient than multiple ADD >commands. So I think we will go with the PUT method. While PUT should currently be the fastest method, you should know that in 1.8a you still had LSVPUTL XEDIT. It's a tradeoff between the slowness of XEDIT + REXX and not running through the process that wasn't scaling up well past a couple thousand subscribers. If you install FIX18AX2 all that junk goes away. Ah, the job just finished. It took about 16 minutes, and the output file is 1.4M. It would clearly be better to send a dozen smaller jobs. Anyway PUT should be a lot faster since it works on the assumption that the list is being replaced. PUT loads everything in memory and then checks for duplicates and the like entirely in memory, to avoid disk I/O. ADD assumes you're going to be adding one user or maybe a dozen, and that it will be faster to read the file every time. It also does a lot more syntax checking and parsing than PUT, because the input address format is much more flexible. ADD is not really meant to be used with 10k entries. Eric