Wed, 27 Dec 2006 09:15:35 -0500
|
> Once upon a time, I had to do something similar. I moved the logs to our
> research computing machine and wrote a multithreaded perl script to do
> the scanning and analysis. Using 6 of the 8 processors, the job took ~4
> days to run, and that was with only about 10 months of logs. Hope your
> site doesn't see too much volume, and with 2 years of logs to search, I
> hope you have some cpu cycles to spare
How big were the log files you were looking at?
Since the goal of the original request was to see the last post in each
list, could you have jsut done "tail -50" (or whatever number would mnake
sense) on each log and processed the results of that rather than looking
at the beginning of the log since that info would be superceded by
subsequent lines in the log?
|
|
|