On Thu, 08 May 1997 13:45:54 -0400 Brad Knowles <[log in to unmask]> said: > The problem is that the Internet side of the equation is not the >limiting factor here. You asked me if I could do better on the Internet side of the equation, not on the internal side. I know nothing about your internal mail system and am not in a position to make comments about it. > I'm confident it could handle twice or even three times that load >without too much difficulty, if there weren't problems elsewhere. *shrug* Likewise, our servers can handle more mail than they are handling today (they are far from saturation). I don't see how this is relevant to the discussion. > Typically, we recieve about twice as much as we send. That's still a >boatload of mail that we send, but since the expansion factor per >outbound email message is low (on average, about two recipients per >message), we don't get your economies of scale with hundred or thousands >of recipients from a single message all served by the same set of MXes, >etc.... I believe I did tell you that our average number of RCPT TO: is on the order of 2. I don't doubt that if you look at a whole day's of activity, somewhere you'll find one transaction that had hundreds or possibly even thousands of recipients served through the same MX. But, on the average, we have two recipients per transaction. Very few of our mailing lists have thousands of recipients to begin with, let alone thousands at the same host. > Not exactly. Since virtually all mail that is sent is transmitted as >soon as it is recived by the other end, it makes very little use of >connection caching. You are again thinking in sendmail terms. This simply does not apply to LSMTP. > Since you've got extremely large economies of scale due to large >numbers of recipients typically served by a set of MXes, Again I don't. >and you've surely optimized your delivery to make maximum use of >connection caching, the actual number of SMTP connections you make is >actually probably quite small. This is correct and is one of the reasons why LSMTP outperforms sendmail. However you would also get this economy with a large workload comprised of a lot of independent messages. > Making SMTP connections is relatively cheap, since you can choose >whether or not to have another queue runner fork off Fork? What is that? :-) > Since receiving mail is inherently interrupt-driven, and you can't >force the other end to make use of connection caching (you can only sit >there and wait to accept multiple messages per connection if the other >end chooses to send them that way), what I want to count is the total >number of SMTP connections you receive per day. The total number of SMTP connections I receive per day is limited by the intrinsic nature of my workload and not by the capacity of the server. This is a bit like asking me how many concurrent processes LSMTP can support, well it supports one, so sendmail "wins". What you really want to know however is how much AOL-style mail LSMTP could pump in and out and on what type of box, and the correct approach is to go through the specs/bid/prototype process and not go on making meaningless comparisons between software with completely different design and internal structure. > I will pass on to the mail systems development management that you >want to re-write the AOL Internet mail gateway system using LSMTP. If >that's something we can do in parallel with our other efforts (and >without a great deal of support required to teach you how the back end >works), then they might be willing to listen. We can look into that as well if it makes sense (I just don't have the information to have an opinion on whether it does or not), but to clarify, this is not what I was proposing. I was thinking along the lines of having LSMTP front end your existing system. It would bear the brunt of all the incoming connections, do basic filtering (reject mail that is not for AOL users, or whatever your policies are), etc. Then it would pass the mail back to your existing unix servers, using the exact number of connections of your choice, which naturally would be cached. Your unix servers would run with a controlled, tunable number of sendmail processes and would not have to open/close connections all the time. You would keep your more advanced filtering tools on sendmail. Likewise, outbound mail would be sent to the LSMTP servers through a sendmail.cf rule and your sendmails would not develop a queue. In other words, your unix systems would work optimally and you could retain your existing interface to the internal system, whatever this may be. This is clearly not the perfect solution, but my goal was to minimize the amount of development work on AOL's side. Basically, you don't need any new code at all, just a small sendmail.cf change. > He keeps remarking on occasion how much mail could be handled by VM >SMTP, but he's changed his tune a bit since we found a bug in that code >with regards to the way it handles MX RRs. I'm going to make a very rough exaggeration here, but it wouldn't be entirely wrong to say that LSMTP is a refined version of the VM SMTP design, ported to affordable hardware and implemented in a more efficient way (fully asynchronous I/O, etc). Back when Matt left IBM, the VM SMTP outperformed sendmail easily on slower machines. Unfortunately development more or less froze when Matt left and the software has not been improved since then. Nevertheless, it did scale to 1.1M daily deliveries (highest recorded figure). The VM SMTP is not implemented very efficiently but is remarkably scalable, given the workloads that were actually being processed at the time it was developed. Eric