[mobile-devel] And Bible indexes and foreign modules

[Martin Denham]

While working on index download for the next release of And Bible I have had
some questions and would be grateful for any input either from the Lucene
perspective or from perspective of doing searches in foreign language
modules, especially Chinese.

The current state of code in svn is that you can download search indexes
from the same repository used by PocketSword but the option to generate an
index is still available.  The main question I have is with regard to
potential compatibility problems using clucene indexes in And Bible.  A
related issue is the size of the jar required to support Chinese searches.

Compatibility Questions:

   - Downloadable indexes were generated by clucene (C+) not Lucene
   (JSword).  To what extent are they compatible?
   - The existing And Bible index code uses the language specific Analyzer
   as defined by JSword.  All my (English) tests show that clucene indexes work
   fine but I can't test foreign language searches.
   - During initial development And Bible could not reliably use
   BibleDesktop (JSword app) generated indexes.  I still don't know why.
   - Is there a way to find which Analyzer was used to generate a specific
   index because the same Analyzer must be used for creation and searching.
   - I have not actually seen any search problems for a long time using
   either clucene or AndBible generated indexes but I only use English modules.

Chinese search support

   - The Chinese Analyzer is half the size of the whole of And Bible.
    Removing it would have a large affect on app size.  What affect does it
   have on searches?
   - Difficult to test foreign language searches - especially Chinese.

I have had a couple of thoughts.  The first may be the way of least
technical resistance.

 *Approach 1*
Use clucene generated indexes and ignore language specific Lucene analyzers.
 I think Sword does this.
Advantages

   - Much smaller application - important on a mobile but less important now
   Android 2.2 can run apps from SD card
   - Index download repository already exists

Disadvantages

   - How good is foreign language searching using clucene indexes?  I can't
   tell.

*Approach 2*
Use JSword Lucene language specific indexes.
Advantages

   - Better search support for foreign languages.  Can somebody give
   examples of difference?

Disadvantages

   - Android apk application file is twice as large
   - Need to create a new repository of JSword generated indexes

Thanks
Martin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.crosswire.org/pipermail/mobile-devel/attachments/20101102/b254bd38/attachment.html>