[sword-devel] SwordReader PocketPC and SmartPhone CAB files.

[DM Smith]

On Jul 8, 2008, at 9:46 AM, Ben Morgan wrote:

> There are two things to note here:
> a) the ESV is a compressed module (uses the zText driver).
> The zText driver is quite a bit slower than the rawtext one (around  
> 20-30x slower!)
> Here are the times taken to getRawEntry for a compressed and  
> uncompressed ESV:
> uncompressed: 1.89 seconds (this is a good length of time)
> compressed: 40.461 seconds (this isn't)
>
> This is a tad depressing - especially since all new modules seem to  
> be being compressed.

I think something else is going on here.

BTW, we compress primarily for the sake of storage on a CD and I think  
also for obfuscation of the source to encourage participation in  
development.

FYI,

The method of storage is double indirection to the text.

The first index gives the start and size of the compressed block.  
Typically, there are 66 (i.e. chapter compression)

The second index gives the start and size of the raw entry in the  
uncompressed block. This is the same as if it weren't uncompressed.

So there are two additional things here that can affect the  
performance in getting raw text:
1) Another lookup in an index.
2) Uncompression of a block. (Note: Uncompression take place in core.)

I don't know about the SWORD engine, but JSword holds the last  
uncompressed block on the premise that the next verse will be  
retrieved and that it is in the same block. With this there will be 66  
uncompressions. If it doesn't then there will be one per verse.

Since the uncompression algorithm is streaming, it would be possible  
to uncompress until the desired start and length were uncovered. Even  
the partial uncompress could be cached and a subsequent call to a  
later verse could pick up where the uncompression left off. As far as  
I know SWORD does not do this for either ZIP or LZSS. JSword does not.

It would be interesting to see if LZSS is any faster than ZIP  
compression.

In Him,
	DM