[jsword-devel] Lucene upgrade

[Sijo Cherian]

This new pull request is intended as a new branch:
https://github.com/crosswire/jsword/pull/82

This pull-request includes code changes to use the new index/query api in
Lucene 4.8.0.
(I called jsword release.version = 2.0.1-luceneupgrade-alpha)
It uses LUCENE_30 version (set in
IndexMetadata.LUCENE_IDXVERSION_FOR_INDEXING) in IndexWriter/QueryParser
for compatibility.
After a transition phase, I am thinking we will switch this version to
latest (to use newer features, less RAM usage in newer index format).

As far as I could test, it seem to be back compatible to existing index
when using LUCENE_30. We need folks to test this for european/asian
language bibles using existing index (since English is using
SimpleLuceneAnalyzer with no stemming etc it is unlikely to have issues in
English bible index).

All feedbacks/testing efforts are very much appreciated.
/sijo




On Thu, May 8, 2014 at 12:36 AM, Sijo Cherian <sijo.cherian at gmail.com>wrote:

>
> I started with changing jsword usage of Lucene api to latest in 4.x (I
> think this is the easier piece & I am close to 75%).
>
> Issue is with existing index upgrade. Currently we're using Lucene 3.0.3,
> but index is build with 2.9. The 4.x lets you read older version index
> upto 3.0. If I can figure out difference between 2.9 vs 3.0 version, then
> we can decide if a hop to lucene 3.6 is necessary. I want to dig a little
> more into it, before a full blown discussion inside this version universe.
>
> I am hoping to provide a transition phase (using config or two plugin
> option), so that existing index is not forced to upgrade. If we keep
> content of current field unchanged, it will reduce one variable.
>
> Meanwhile if you guys can test the indexversion upgrade pullreq (
> https://github.com/crosswire/jsword/pull/79) & see if it meets the
> requirements of atleast your respective app's index upgrades, that will be
> great.
> In AndBible's usecase, it should help to prompt user to rebuild/download
> index of the one book (that is being search).
>
> /sijo
>
>
> On Mon, May 5, 2014 at 3:17 PM, Chris Burrell <christopher at burrell.me.uk>wrote:
>
>> Hi
>>
>> Which version are we going for?
>>
>> Not sure what you mean by 'best not to change the index structure'? Do
>> you mean best to keep the current fields indexed with the current content?
>> (if so, I agree, let's do this a step at a time).
>>
>> Chris
>>
>>
>>
>> On 3 May 2014 21:32, Sijo Cherian <sijo.cherian at gmail.com> wrote:
>>
>>> FYI
>>> I just started working on upgrading code to newer lucene version, &
>>> keeping back compatibility.
>>>
>>> More updates after some progress.
>>> It is best to not change index structure in default plugged-in version,
>>> for now.
>>> /sijo
>>>
>>> _______________________________________________
>>> jsword-devel mailing list
>>> jsword-devel at crosswire.org
>>> http://www.crosswire.org/mailman/listinfo/jsword-devel
>>>
>>>
>>
>> _______________________________________________
>> jsword-devel mailing list
>> jsword-devel at crosswire.org
>> http://www.crosswire.org/mailman/listinfo/jsword-devel
>>
>>
>
>
> --
> Regards,
> Sijo
>



-- 
Regards,
Sijo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.crosswire.org/pipermail/jsword-devel/attachments/20140511/0456efe3/attachment.html>