I made a few more changes to the code to further improve the performance
of small jobs, and I also found a list with 8543 recipients which I was
authorized to review, so I now have figures for very large lists. As I
expected, the performance boost exceeds 20 with a list of that size. Here
are the new figures:
+-----------------+--------------+---------------+----------+
| Job type | Ratio (TCPU) | Absolute TCPU | Formerly |
+-----------------+--------------+---------------+----------+
| Small/relayed | 3.27 | 0.658 & 0.201 | 0.274 |
| Small/generated | 3.32 | 0.687 & 0.207 | 0.284 |
+-----------------+--------------+---------------+----------+
| Large/relayed | 9.20 | 5.244 & 0.570 | 0.647 |
| Large/generated | 9.28 | 7.967 & 0.858 | 1.136 |
+-----------------+--------------+---------------+----------+
| Huge/relayed | 40.39 | 230.2 & 5.7 | No data |
| Huge/generated | 22.62 | 312.1 & 13.8 | No data |
+-----------------+--------------+---------------+----------+
The "huge" job has 17 times as many recipients as the "large" one, with
70% of recipients passed on to other servers and 30% delivered locally
via BSMTP. As you can see the new code is about linear, whereas the old
code quickly gets out of control. Anyway if you run a list of that size,
you now have a real incentive to beta-test the new code :-)
When ordering the code from me you will receive a series of UPD17F
shipment files *without JCL*. They are to be CARD LOADed manually. Once
this is done you will have both DISTRIBUTE processors and the old one
will remain active by default; the new one will be available for testing
under the name DIST3. To run the new code in production you simply rename
LSVDIST2 EXEC to some other name and copy LSVDIST3 EXEC to LSVDIST2 EXEC
(or just rename it if you don't mind getting errors from DIST3). The
DIST3 command verb is a temporary addition for testing the new code and
will be removed in the final 1.7f shipment. The algorithms and protocols
are the same, the only thing which has changed is the implementation;
there is no reason to rename the command as when the old DISTRIBUTE
command was replaced with DIST2 many years ago.
When the old DISTRIBUTE is active, the output of SHOW DISTRIBUTE and SHOW
STATS are partly incorrect. This is because the two processors don't keep
their statistics in the same place, and I didn't want to waste time
dual-pathing LSVSHOW, especially as it isn't very easy to figure out
which one is active (in fact if you make the DIST2 verb point to LSVDIST3
in STDCMD FILE both will be active - the new code for relayed jobs from
the network and the old code for internally generated ones).
There are small differences in format in the jobs produced by the two
processors. For instance, the 'TO user1 user2 user3...' syntax is very
inefficient for large distributions as it requires a very long string of
contiguous storage, so the new code never generates it (it still accepts
it of course). Some messages have changed, some abnormal conditions are
handled differently, and so on. In other words, please report anything
which looks like a problem, even if you're not sure, but don't report
something only because it isn't exactly like it used to be.
Finally, the information you get when you specify DEBUG=YES has changed.
Having the actual recipients listed was a serious inconvenience for large
lists and left little space for the hostname part of the address, which
is the most important. The new format shows only the host names, and
duplicates are removed. When there is only one recipient for that
particular host, 'single recipient' is shown in the last column. This may
or may not modify the routing decision, but it would have been difficult
to change the code to display this message only when it actually impacts
the routing.
Eric
|
