[jsword-devel] Lucene searching improved

[Joe Walker]

There are probably some memory savings we could make to Bitwise
passage, feel free to make them. For me I suspect that there are more
important fixes, but we could probably do with using less memory all
the same.

Joe.

On Mon, 31 Jan 2005 19:13:07 -0500, DM Smith <dmsmith555 at yahoo.com> wrote:
>  Joe,
>      I thought that renaming the jdom.jar would force a rebuild. It did not.
> I took a peek on crosswire
>  to see if I could do a clean build and you own the build and it's files so
> you will have to do a clean build.
>      By the way can you add me to the jsword group? Some jsword directories
> do not have world read.
>      As to BitwisePassage, I think it is a bit heavier than it needs to be.
> It takes up lots of memory. And
>  by and large it is a sparse array. It could stand some space optimization.
> For example, only hold enough
>  bits for the verses from the first to the last in the answer and have a
> cardinal that is added to the bit position
>  to give the verse. Subtraction would go from the verse to the bit position.
> Bit operations between two
>  BitwisePassages would be complicated by the need for normalization.
>      If answers in a BitwisePassage were sparse between the first and last
> verse, then a collection of BitSets
>  could be used.
>      I had thought of this earlier, but felt that there were bigger problems
> to solve.
>  Later,
>      DM
>  
>  Joe Walker wrote: 
>  Thanks DM, this all looks like the sensible thing to do. I've been mulling
> over the PassageTally thing, but I've not worked out a reason why it was as
> it was yet. I need to get my head around how it affects best match searching
> properly, but since it fixes a very clear problem it sounds good to me. On
> RocketPassage - it certainly used to be the fastest; I spent some time ages
> ago tweaking it to include the best of the other Passage implementations,
> however since then you've fixed a few issues in BitwisePassage (IIRC) so it
> may well no longer be optimal. Maybe I was engaging in a bit of premature
> optimization! Joe. On Wed, 26 Jan 2005 21:31:09 -0500, DM Smith
> <dmsmith555 at yahoo.com> wrote: 
>  Hi, I have fixed the bug in the search where bread returned 20 results
> rather than over 350. The problem was that it was using a PassageTally to
> collect the results. I changed the code to call book.createEmptyKeyList().
> This uses the user's choice of Passage. By default it is set to
> RocketPassage. Is there a better default? I also changed the indexing to
> save the verse reference as UnIndexed. As it was the verse was being indexed
> and stored. I didn't see a need to index the verse as it is indexed already.
> And the search logic never searched against that Field. I also changed
> BookData to skip the children of the <div> element that are not <verse>
> elements. The call to getPlainText still gets and indexes the non-verse text
> in the <verse> element. Gen 18.5 in the KJV is an example. In His Service,
> DM _______________________________________________ jsword-devel mailing list
> jsword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/jsword-devel
> _______________________________________________ jsword-devel mailing list
> jsword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/jsword-devel