[sword-devel] size of xapian indices

[Greg Hellings]

Another possibility to add to our comparison might be http://lucy.apache.org

It appears active (latest release within the past two months). They freely
state they aren't as fast yet as Lucene is, but it might be faster than
Xapian's three and a half hours for Karl.. Might be worth consideration.

--Greg
On Nov 27, 2014 11:25 AM, "Troy A. Griffitts" <scribe at crosswire.org> wrote:

> Hey Karl,
>
> I don't think we're at a place where we want to promote Xapian as the
> default and recommended indexed search framework. We added support for it
> due to CLucene's lack of updates, but support is very preliminary and we
> haven't done any speed or size testing with it until your report! :) Maybe
> there are options to optimize the creation time and index size. Thanks for
> the report!
>
> Troy
>
> On November 27, 2014 8:59:29 AM PST, Karl Kleinpaste <karl at kleinpaste.org>
> wrote:
>>
>> I hadn't thought too much about the new Xapian search usage until I had
>> reason this morning to look at disc usage under ~/.sword.
>>
>> Xapian indices are monstrous in size.
>>
>> A touchstone for this is that my personal NETnote module has rawcom ot+nt
>> files whose size total about 23M.  But the Xapian indices for the module
>> are 140M.  Until a little while ago, ~/.sword contained about 28G, of which
>> 23.2G was Xapian indices...  Then I deleted all Xapian indices and am back
>> to CLucene using --without-xapian in Sword.
>>
>> I realize the reason for Xapian is CLucene's lack of support.  But do we
>> really want a 7-fold size increase in module space occupation for the sake
>> of the new indexer/search?
>>
>> ------------------------------
>>
>> sword-devel mailing list: sword-devel at crosswire.org
>> http://www.crosswire.org/mailman/listinfo/sword-devel
>> Instructions to unsubscribe/change your settings at above page
>>
>>
> --
> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>
> _______________________________________________
> sword-devel mailing list: sword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.crosswire.org/pipermail/sword-devel/attachments/20141127/017d11a1/attachment.html>