[jsword-devel] Fwd: [jsword] JS-98: IndexManager.needsReindexing(Book book); as the API to detect if per-book index need to be recreated (#74)

Sun Apr 6 14:17:20 MST 2014

Thanks Chris.

Is the only usecase for "external Lucene indexes" to install a downloadable
index?

I am assuming any "external Lucene indexes" usage is going through
IndexManager interface (in our default case LuceneIndexManager).
In that case I will add version management through IndexManager api.

So the Installed index versions will go in the folder returned by
LuceneIndexManager.getStorageArea(Book book)

cheers back
sijo

On Sat, Apr 5, 2014 at 5:24 PM, Chris Burrell <chris at burrell.me.uk> wrote:

> Sorry about the radio-silence. I agree with the thread so far. STEP also
> uses that folder to deposit its own set of Lucene indexes. So that would be
> a nice central place.
> '
> 'twould be a nice feature if we could register external Lucene indexes
> with JSword such that an external app could use Sijo's work. In other
> words, for other indexes, external to JSword, it would be nice to be able
> to ask whether indexes could be upgraded, etc.
>
> Cheers,
> Chris
>
>
>
> On 2 April 2014 23:34, DM Smith <dmsmith at crosswire.org> wrote:
>
>>
>> On Apr 2, 2014, at 5:23 PM, Sijo Cherian <sijo.cherian at gmail.com> wrote:
>>
>> I will appreciate feedback or concern from jsword users (And,BD..) on
>> having a property file to store current index version (currently indexed
>> default version + per-book's index version, if different).
>>
>> After that, any recommendation on suitable location for this file (I am
>> thinking {IndexFolder}/JSword/lucene/js.index.metadata.prop file). I am
>> not sure if all platforms have same folder structure (like JSword/lucene)
>>
>>
>> I think it should be in the folder that holds the lucene index files. I'm
>> not sure if AndBible has done anything special to the code that locates the
>> lucene indexes, but the JSword code that determines the state of the index
>> presumes the location. It differs by platform, but that is transparent
>> within JSword. I'm going to guess that AndBible has handled pathing so that
>> it also is transparent. So I think that if you can put it there all
>> front-ends should work with it.
>>
>> note: In code comments, I am referring to this file as
>> InstalledIndex.prop.
>>
>>
>> Regarding the writing and retrieval of values it is up to you. You might
>> be able to add the API to IndexManager.
>>
>> It seems that IndexManager should have a IndexStatus getIndexStatus().
>> (search for setIndexStatus and you'll see why). Ultimately a user asks
>> getIndexStatus from Book, which delegates the question to BookMetaData. The
>> status INVALID is defined as the index is no longer valid. Nothing sets
>> this. Nothing really uses it either.
>>
>>
>> Many thanks,
>>
>> sijo
>>
>>
>>
>> On Sun, Mar 30, 2014 at 4:27 PM, DM Smith <dmsmith at crosswire.org> wrote:
>>
>>> Sijo asked this via github. Thought it'd be good to get feedback here.
>>>
>>> -- DM
>>>
>>> Begin forwarded message:
>>>
>>> *From: *Sijo Cherian <notifications at github.com>
>>> *Subject: **[jsword] JS-98: IndexManager.needsReindexing(Book book); as
>>> the API to detect if per-book index need to be recreated (#74)*
>>> *Date: *March 29, 2014 at 7:31:52 PM EDT
>>> *To: *crosswire/jsword <jsword at noreply.github.com>
>>> *Reply-To: *crosswire/jsword <
>>> reply+i-30455246-2897689b705296a200fa088d2af792bed6bc6a4a-1645068 at reply.github.com
>>> >
>>>
>>> Following is the upgrade criteria implemented by comparing IndexMetadata
>>> from prop files: needsReindexing returns true, if Latest.Index.Version.xxx
>>> > Installed.Index.Version.xxx OR if {index folder} is not found at all.
>>>
>>>    - For now uses two prop file: 1. IndexMetadata.prop file (can be
>>>    updated with each release of JSword) 2. InstalledIndex.prop file for
>>>    "Installed.Index.Version.xx" values that should stay persistent on clients
>>>    computer between upgrades
>>>    - Support for PerBookVersion property: Since index creation api is
>>>    per Book based, added support for (optional) PerBookVersion properties to
>>>    allow selective upgrade of one book's index, if needed.
>>>    - This pull request, still assumes index version as 1.2, so should
>>>    not break anything
>>>
>>> Notes: Looked into storing version values in the lucene index using
>>> FlexibleIndexing in lucene 4.0, but that seem very dependent on lucene api.
>>> Prop file gives human readable configuration and easy to debug.
>>>
>>> ToDo/Questions:
>>> 1. Managing the add/update of values in InstalledIndex.prop
>>> programmatically
>>> 2. Need recommendation on suitable location for InstalledIndex.prop file
>>> (perhaps as {IndexFolder}/JSword/lucene/js.index.metadata.prop file)
>>> 3. Need to add reindexAllBooksIfNeeded() API, that folks (AndBible, BD,
>>> Alkitab and?) can use to upgrade all index as bulk, for the installed books
>>> , only then the DefaultInstalledVersion can be updated.
>>> o.c.j.bridge.BookIndexer.java shows a sample API in the comments.
>>> 4. How do the clients want the DownloadIndex option to work with index
>>> versioning?
>>> ------------------------------
>>> You can merge this Pull Request by running
>>>
>>>   git pull https://github.com/sijocherian/jsword master
>>>
>>> Or view, comment on, or merge it at:
>>>
>>>   https://github.com/crosswire/jsword/pull/74
>>> Commit Summary
>>>
>>>    - JS-98 : mechanism to detect if index need to be recreated by
>>>    comparing IndexMetadata from prop file for now Started partially with the
>>>    patch from Tonny Kohar dated 15/May/09
>>>    - - Added InstalledIndex.prop file (location of this file TBD)
>>>
>>> File Changes
>>>
>>>    - *M* src/main/java/org/crosswire/jsword/bridge/BookIndexer.java<https://github.com/crosswire/jsword/pull/74/files#diff-0>(20)
>>>    - *M* src/main/java/org/crosswire/jsword/index/IndexManager.java<https://github.com/crosswire/jsword/pull/74/files#diff-1>(16)
>>>    - *M*
>>>    src/main/java/org/crosswire/jsword/index/lucene/IndexMetadata.java<https://github.com/crosswire/jsword/pull/74/files#diff-2>(29)
>>>    - *A*
>>>    src/main/java/org/crosswire/jsword/index/lucene/InstalledIndex.java<https://github.com/crosswire/jsword/pull/74/files#diff-3>(67)
>>>    - *M*
>>>    src/main/java/org/crosswire/jsword/index/lucene/LuceneIndexManager.java<https://github.com/crosswire/jsword/pull/74/files#diff-4>(53)
>>>    - *M* src/main/resources/IndexMetadata.properties<https://github.com/crosswire/jsword/pull/74/files#diff-5>(18)
>>>    - *A* src/main/resources/InstalledIndex.properties<https://github.com/crosswire/jsword/pull/74/files#diff-6>(39)
>>>    - *A*
>>>    src/test/java/org/crosswire/jsword/index/lucene/LuceneIndexManagerTest.java<https://github.com/crosswire/jsword/pull/74/files#diff-7>(131)
>>>
>>> Patch Links:
>>>
>>>    - https://github.com/crosswire/jsword/pull/74.patch
>>>    - https://github.com/crosswire/jsword/pull/74.diff
>>>
>>> --
>>> Reply to this email directly or view it on GitHub<https://github.com/crosswire/jsword/pull/74>
>>> .
>>>
>>>
>>>
>>> _______________________________________________
>>> jsword-devel mailing list
>>> jsword-devel at crosswire.org
>>> http://www.crosswire.org/mailman/listinfo/jsword-devel
>>>
>>>
>>
>>
>> --
>> Regards,
>> Sijo
>> _______________________________________________
>> jsword-devel mailing list
>> jsword-devel at crosswire.org
>> http://www.crosswire.org/mailman/listinfo/jsword-devel
>>
>>
>>
>> _______________________________________________
>> jsword-devel mailing list
>> jsword-devel at crosswire.org
>> http://www.crosswire.org/mailman/listinfo/jsword-devel
>>
>>
>
> _______________________________________________
> jsword-devel mailing list
> jsword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/jsword-devel
>
>

-- 
Regards,
Sijo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.crosswire.org/pipermail/jsword-devel/attachments/20140406/3b88da94/attachment.html>