I have just became fed up with EXECIO's poor performances and above all with
the philosophical problem, should I attempt to take advantage of the release 4
'STEM' option? This does save a few CPU  cycles but it forces me to write code
to  'emulate'  the STEM  processing  under  SP3 --  this  costs  me a  lot  of
programming time  and there  is no way  I can  do it as  a separate  exec, the
routine must therefore but included and customized in the program every time.
 
  I have therefore decided to spend a  few hours on an EXECIO substitute, once
and for all. My  goal was to make it 5 times faster  for program stack output,
and 3 times faster for STEM output (EXECCOMM eats up an enormous amount of CPU
time in both cases so...) This program can only read a WHOLE file, from top to
bottom, and always FINIShes it -- that's  restrictive but that's what I end up
doing 95% of the time  in the code and it's the only way  I can use faster I/O
procedures.
 
  Here are a few stats. I tried the stuff both on a RECFM V file (my favourite
EARNTECH  NOTEBOOK, 9984  records) and  a RECFM  F one  (a TEXT  file of  3918
records).
 
A) RECFM V file:
 
    - Time required for REXX to read up the data from the program stack:
 
          --> 5.9 seconds
 
    - EXECIO to program stack: 5.9 seconds
 
    - EXECIO to stem: 10.6 seconds (better than 5.9+5.9)
 
    - LSVFILER to program stack: 1.05 second (1.05+5.9 better than 10.6)
 
    - LSVFILER to stem: 2.9 seconds
 
 
B) RECFM F file:
 
    - Time required for REXX to read up the data from the program stack:
 
          --> 2.5 seconds
 
    - EXECIO to program stack: 2.5 seconds
 
    - EXECIO to stem: 4.4 seconds
 
    - LSVFILER to program stack: 0.5 second
 
    - LSVFILER to stem: 1.3 second
 
 
  LSVFILER is therefore 5.6 times faster  on program stack reads and 3.7 times
faster on stem reads. That's better than the  5 and 3 I wanted to achieve, but
I knew it was  difficult to do better. Actually LSVFILER reads  the file in 10
times  faster than  EXECIO,  but must  still  spend the  same  amount of  time
queueing it to the program stack or  EXECCOMMing it. The STEM option works for
both  VM/SP  4  and VM/SP  3  of  course,  so  I  can  now choose  to  use  it
unconditionally :-) :-) :-)
 
  Brief technical  description: LSVFILER  reads in  the file  in a  single I/O
operation, and will then output its contents  one record at a time. Should CMS
run out of virtual storage, LSVFILER would immediately release as much storage
as possible from the buffer housing the contents of the file and make a retry.
If  the retry  fails, an  error is  reported to  the caller.  If the  retry is
successful, no error is reported (but CMS will type an INSUFFICIENT STORAGE TO
SATISFY DMSFREE CALL FROM <whatever>; this  message is issued as a consequence
of the DMSCAT  call and cannot be  suppressed). If no storage  can be released
because nothing in  the buffer has been transferred to  the program stack yet,
an error condition  is reported. The same applies for  STEM output processing.
Additionally, when STEM output is required,  LSVFILER will obtain a 16k buffer
at initialization to act as a batch for EXECCOMM calls. If a 16k buffer cannot
be obtained, LSVFILER will take what the  system can give it and work on this.
As EXECCOMM  requests are built  up, they are batched  in the buffer  until it
becomes  full. Any  request larger  than the  buffer is  directed to  EXECCOMM
without causing the buffer to be flushed. DMSCAT is called via BALR instead of
SVC 202. etc, etc. I don't think  it could be made much faster, in particular,
increasing the buffer size to 32k or  64k did not yield any noticeable result.
The only way to speed the thing up  is to optimize DMSFRE, but this is another
matter entirely... :-)
 
  Anyway the thing will be part of release 1.5j of LISTSERV, and I will slowly
rewrite the various  execs to make use of LSVFILER  (STEM whenever possible. I
would be  very interested  in non-LISTSERV  beta-testers as  I never  feel too
confident  with new  MODULEs, especially  those which  seem to  work perfectly
after the  first assembly...  :-) I  am going to  make the  module TEMPORARILY
available from my LISTSERV (TELL LISTSERV  AT FRECP11 GET LSVFILER MODULE), so
anybody can retrieve it freely.
 
  Ah, I was going to forget the syntax... It's very simple:
 
LSVFILER fn ft <fm> <(options>
 
options: FIFO LIFO STRIP STEM xxxx.
 
rc: 0 (ok), 1 (truncated -- if output to program stack), 24 (syntax), 28 (file
    not found), 100 (file  is on a CDF disk), 1xxx  (DMSFREE error xxxx), 2xxx
    (read error xxxx), 3xxx (error xxx from DMSCAT or EXECCOMM).
 
The default fm is '*', the default options are: FIFO.
 
Don't expect sophisticated error diagnostics from the program: LSVFILER |1_!%"
\SDF}E› XX,(  will merely tell you  rc=28 (file not found)  without typing any
nasty message on the console :-)
 
  Eric