[jsword-devel] Lucene index help

[Martin Denham]

Wow.  I didn't expect somebody to come straight in with the answer.  Thanks
for that.  It seems to explain the problem very well.

When I get to the pc I was using I will check what version of BibleDesktop
is installed.  Does the latest build of BibleDesktop contain the new
stemming feature that is in the latest JSword code in Subversion?

So, assuming that problem is sorted it would be great if we could create a
repository of the latest JSword indexes and then I can point And Bible at
it.  I am happy to do this, or to assist, or just to use them, but at the
moment I don't think I have write access to any CrossWire repositories.

Kind regards
Martin

On 4 November 2010 16:44, DM Smith <dmsmith at crosswire.org> wrote:

> On 11/04/2010 11:26 AM, Martin Denham wrote:
>
>> Does anybody know any reason why a search for 'blessed' does not return
>> any search results in ESV but searching for 'bless' work perfectly?
>>
>
> As best as I can tell it is a miss match between the index and the library.
> I just did the same search as you and got the same results as you. Then I
> deleted the index and rebuilt it. Then it started to work.
>
>
>
>> When I download  BibleDesktop (JSword) generated indexes to And Bible I
>> have noticed that some searches like 'blessed' stop working but I can't
>> figure out what the problem is and would appreciate some pointers as to
>> areas to look.
>>
>
> It is critical that the same jsword.jar is used to build the index and to
> search it.
>
> In this case the problem is that stemming has been introduced in the newest
> version of JSword. This invalidates old indexes, but there is not a
> mechanism to know that. Well, there is but it is not complete.
>
>
>
>> I have checked that the correct Analyzer is being used but I am not sure
>> what else to check or if the 'blessed'/'bless' issue might point to a
>> specific problem area.
>>
>
> The analyzer is merely a chain of a tokenizer and a bunch of filters.
>
>
>
>> The plan is to download pre-created indexes to And Bible and in theory
>> those indexes should be generated by JSword but currently And Bible can only
>> use indexes it creates itself or which have been created by CLucene/Sword.
>>
>
> The indexes that CLucene/Sword create are not compatible with JSword and
> haven't been for many releases. (And visa versa). For the most part they
> work for English, but as you found out "most part" isn't good enough.
>
> The biggest problem is that CLucene development is stagnant and far behind
> that of Java Lucene. The second problem is that unless a versioning
> mechanism is added to SWORD, the SWORD indexes will not improve or gain new
> features.
>
> We improved JSword w/o a versioning mechanism and are suffering the
> problems. It needs to be fixed with the next point release.
>
>
>
>> All advice, opinions, and comments are appreciated.
>>
>> Many thanks
>> Martin
>>
>
> In His Service,
>    DM
>
> _______________________________________________
> jsword-devel mailing list
> jsword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/jsword-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.crosswire.org/pipermail/jsword-devel/attachments/20101104/88aaf947/attachment.html>