[sword-devel] engine profiling

[Troy A. Griffitts]

With the recent snubs at the engine speed :), I've been doing some 
profiling to find out when things are fast and when things are slow.

I've done some easy and quick minor speedups, but here are my 
conclusions.  The numbers are later:

Filters take time, but not as much as we had previous thought.  There is 
room for them to be optimized more.

It seems the real speed issue is in the zip compress driver (as someone 
recently mentioned here).  The difference between iterating the KJV 
(heavily tagged OSIS zText) and GerLut (reasonably tagged GBF RawText) 
without filters involved, was about 5x speed difference (12.9s vs. 2.7s).

Difference between KJV with and without filters does not even double the 
time (12.9s vs. 20s).

Now, if we were using a RawText driver and getting a 2.7 iteration time 
without filter, then adding 8 seconds for filtering would be a 
significant speed hit (~3x), but we're not.

So, basically, in our current configurations using a compressed driver, 
if we only concentrate improvement efforts on filtering (soText; which 
is good and can potentially removed 8 seconds of processing on a 
desktop) we won't even cut the iteration time in half.

So, we can tackle this on two fronts.  I will have a look at the 
compression driver and see if there is anything I can do to make things 
zing in there.

Here are the tests boiled down:


[scribe at laptop cmdline]$ time ./outplain GerLut > gerlut.txt
real	0m2.707s
user	0m2.082s
sys	0m0.557s

[scribe at laptop cmdline]$ time ./outplain KJV > kjv.txt
real	0m12.900s
user	0m11.947s
sys	0m0.623s

ADDED SWBASICFILTER loop back in

[scribe at laptop cmdline]$ time ./outplain KJV > kjv.txt
real	0m13.250s
user	0m12.344s
sys	0m0.723s

ADDED OSISPlain back in

[scribe at laptop cmdline]$ time ./outplain KJV > kjv.txt
real	0m16.638s
user	0m15.614s
sys	0m0.824s

ADDED ALL KJV FILTERS BACK IN

[scribe at laptop cmdline]$ time ./outplain KJV > kjv.txt

real	0m20.055s
user	0m19.068s
sys	0m0.772s