[jsword-devel] Fwd: Search Index Downloading

[Troy A. Griffitts]

Hey guys,
 	I'd like to do some experiments to see if clucene and Java Lucene 
indecies are binary compatible.

 	I also like the idea of a subdirectory under idx for keeping 
different kinds of indecies.  I might suggest even 1 more level under L1, 
if you are planning for version changes of your index structure.

 	e.g. C++ SWORD supports a pluggable index architecture, and we are 
hoping to write some cool indexers for morphologically declined searches, 
etc.  We could keep pre-generated index sets under different 
subdirectories under idx for each plugin.

 	On the downside, we release updated modules on a regular basis-- 
some modules more 'regular' than others.  To keep the indecies up to date 
for each module should not be the module creators responsibility.  I 
wouldn't expect our current maintainers to run a number of different 
indexers every time they release a new module, unless the process was 
nearly completely automated to handle ALL types of indexing.

 	Up until this consideration, we have always taken the methodology 
of generating anything needed for a plugin on demand on the end user's 
system.  Which is always the least maintenance option for us :)

 	-Troy.


On Mon, 11 Oct 2004, Joe Walker wrote:

> Getting Reply and ReplyAll confused again ...
>
> ---------- Forwarded message ----------
> From: Joe Walker <joseph.walker at gmail.com>
> Date: Mon, 11 Oct 2004 08:12:18 +0100
> Subject: Re: Search Index Downloading
> To: "Troy A. Griffitts" <scribe at crosswire.org>
>
> How about we use /pub/sword/raw/idx/L1/[book].zip then?
> If Java Lucene indexes and CLucene indexes are compatible then it
> won't be proprietary to JSword. If they are not compatible, or if you
> want to use different options in creating the index then you can use
> /pub/sword/raw/idx/C1/[book].zip or something.
>
> Joe.
>
>
>
> On Sun, 10 Oct 2004 22:12:00 -0700, Troy A. Griffitts
> <scribe at crosswire.org> wrote:
>> Hey Joe,
>>         That's fine.  Let me know if there is anything I need to do for you.
>> Don't we have a /pub/jsword directory for your stuff?  I understand what
>> you mean by having the same base directory for modules (which would be
>> /pub/sword/raw for our server, so maybe /pub/sword/raw/idx, but this
>> isn't a sword module data structure.  This is jsword's proprietary (in
>> the sense of not publicly sword declared) data.  It would be nice to
>> unify a common index format for sword modules.
>>
>>         Does it really take lucene 5+ minutes to generate?  That's a bummer.
>> You would think it wouldn't take much longer than a single non-index
>> search thru the Bible.
>>
>>         To belatedly answer your question on sword-devel, I honestly have no
>> idea if clucene indecies are binary compatible with the java lucene
>> counterpart.
>>
>>         -Troy.
>>
>>
>>
>>
>> Joe Walker wrote:
>>> Hi Troy,
>>>
>>> I'd like to allow users of Bible Desktop to download search indexes
>>> because they take about 5 mins to generate. A search index is between
>>> 2-3Mb per book so it ought not to take up too much space.
>>>
>>> Ideally we would use an FTP directory on crosswire something like:
>>> - /pub/sword/search/jsword/L1/[book].zip
>>>
>>> It starts /pub/sword so that if the beta modules site (or other
>>> download sites) come online we can just remember one root path per
>>> module site. The search/jsword bit would keep our stuff from getting
>>> in anyone elses way. L1 is simply a version number so we can update
>>> the index format without huge turmoil.
>>>
>>> Is that OK?
>>> Thanks,
>>>
>>> Joe.
>>
>>
> _______________________________________________
> jsword-devel mailing list
> jsword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/jsword-devel
>