[jsword-devel] Fwd: [jsword] JS-98: IndexManager.needsReindexing(Book book); as the API to detect if per-book index need to be recreated (#74)

Tue Apr 8 06:54:26 MST 2014

Thanks Martin for the layout.

ps: Your mail arrived 2 days late I think, in my mailbox.

On Sat, Apr 5, 2014 at 5:40 PM, Martin Denham <mjdenham at gmail.com> wrote:

> Regarding storage of indexes, And Bible specifies an unusual root
> directory for JSword but beneath that root directory everything is standard
> so I do not foresee any problems.
>
> Here is a screenshot of the layout:
>
> [image: Inline images 1]
> Martin
>
>
> On 5 April 2014 22:24, Chris Burrell <chris at burrell.me.uk> wrote:
>
>> Sorry about the radio-silence. I agree with the thread so far. STEP also
>> uses that folder to deposit its own set of Lucene indexes. So that would be
>> a nice central place.
>> '
>> 'twould be a nice feature if we could register external Lucene indexes
>> with JSword such that an external app could use Sijo's work. In other
>> words, for other indexes, external to JSword, it would be nice to be able
>> to ask whether indexes could be upgraded, etc.
>>
>> Cheers,
>> Chris
>>
>>
>>
>> On 2 April 2014 23:34, DM Smith <dmsmith at crosswire.org> wrote:
>>
>>>
>>> On Apr 2, 2014, at 5:23 PM, Sijo Cherian <sijo.cherian at gmail.com> wrote:
>>>
>>> I will appreciate feedback or concern from jsword users (And,BD..) on
>>> having a property file to store current index version (currently indexed
>>> default version + per-book's index version, if different).
>>>
>>> After that, any recommendation on suitable location for this file (I am
>>> thinking {IndexFolder}/JSword/lucene/js.index.metadata.prop file). I am
>>> not sure if all platforms have same folder structure (like
>>> JSword/lucene)
>>>
>>>
>>> I think it should be in the folder that holds the lucene index files.
>>> I'm not sure if AndBible has done anything special to the code that locates
>>> the lucene indexes, but the JSword code that determines the state of the
>>> index presumes the location. It differs by platform, but that is
>>> transparent within JSword. I'm going to guess that AndBible has handled
>>> pathing so that it also is transparent. So I think that if you can put it
>>> there all front-ends should work with it.
>>>
>>> note: In code comments, I am referring to this file as
>>> InstalledIndex.prop.
>>>
>>>
>>> Regarding the writing and retrieval of values it is up to you. You might
>>> be able to add the API to IndexManager.
>>>
>>> It seems that IndexManager should have a IndexStatus getIndexStatus().
>>> (search for setIndexStatus and you'll see why). Ultimately a user asks
>>> getIndexStatus from Book, which delegates the question to BookMetaData. The
>>> status INVALID is defined as the index is no longer valid. Nothing sets
>>> this. Nothing really uses it either.
>>>
>>>
>>> Many thanks,
>>>
>>> sijo
>>>
>>>
>>>
>>> On Sun, Mar 30, 2014 at 4:27 PM, DM Smith <dmsmith at crosswire.org> wrote:
>>>
>>>> Sijo asked this via github. Thought it'd be good to get feedback here.
>>>>
>>>> -- DM
>>>>
>>>> Begin forwarded message:
>>>>
>>>> *From: *Sijo Cherian <notifications at github.com>
>>>> *Subject: **[jsword] JS-98: IndexManager.needsReindexing(Book book);
>>>> as the API to detect if per-book index need to be recreated (#74)*
>>>> *Date: *March 29, 2014 at 7:31:52 PM EDT
>>>> *To: *crosswire/jsword <jsword at noreply.github.com>
>>>> *Reply-To: *crosswire/jsword <
>>>> reply+i-30455246-2897689b705296a200fa088d2af792bed6bc6a4a-1645068 at reply.github.com
>>>> >
>>>>
>>>> Following is the upgrade criteria implemented by comparing
>>>> IndexMetadata from prop files: needsReindexing returns true, if
>>>> Latest.Index.Version.xxx > Installed.Index.Version.xxx OR if {index folder}
>>>> is not found at all.
>>>>
>>>>    - For now uses two prop file: 1. IndexMetadata.prop file (can be
>>>>    updated with each release of JSword) 2. InstalledIndex.prop file for
>>>>    "Installed.Index.Version.xx" values that should stay persistent on clients
>>>>    computer between upgrades
>>>>    - Support for PerBookVersion property: Since index creation api is
>>>>    per Book based, added support for (optional) PerBookVersion properties to
>>>>    allow selective upgrade of one book's index, if needed.
>>>>    - This pull request, still assumes index version as 1.2, so should
>>>>    not break anything
>>>>
>>>> Notes: Looked into storing version values in the lucene index using
>>>> FlexibleIndexing in lucene 4.0, but that seem very dependent on lucene api.
>>>> Prop file gives human readable configuration and easy to debug.
>>>>
>>>> ToDo/Questions:
>>>> 1. Managing the add/update of values in InstalledIndex.prop
>>>> programmatically
>>>> 2. Need recommendation on suitable location for InstalledIndex.prop
>>>> file (perhaps as {IndexFolder}/JSword/lucene/js.index.metadata.prop file)
>>>> 3. Need to add reindexAllBooksIfNeeded() API, that folks (AndBible, BD,
>>>> Alkitab and?) can use to upgrade all index as bulk, for the installed books
>>>> , only then the DefaultInstalledVersion can be updated.
>>>> o.c.j.bridge.BookIndexer.java shows a sample API in the comments.
>>>> 4. How do the clients want the DownloadIndex option to work with index
>>>> versioning?
>>>> ------------------------------
>>>> You can merge this Pull Request by running
>>>>
>>>>   git pull https://github.com/sijocherian/jsword master
>>>>
>>>> Or view, comment on, or merge it at:
>>>>
>>>>   https://github.com/crosswire/jsword/pull/74
>>>> Commit Summary
>>>>
>>>>    - JS-98 : mechanism to detect if index need to be recreated by
>>>>    comparing IndexMetadata from prop file for now Started partially with the
>>>>    patch from Tonny Kohar dated 15/May/09
>>>>    - - Added InstalledIndex.prop file (location of this file TBD)
>>>>
>>>> File Changes
>>>>
>>>>    - *M* src/main/java/org/crosswire/jsword/bridge/BookIndexer.java<https://github.com/crosswire/jsword/pull/74/files#diff-0>(20)
>>>>    - *M* src/main/java/org/crosswire/jsword/index/IndexManager.java<https://github.com/crosswire/jsword/pull/74/files#diff-1>(16)
>>>>    - *M*
>>>>    src/main/java/org/crosswire/jsword/index/lucene/IndexMetadata.java<https://github.com/crosswire/jsword/pull/74/files#diff-2>(29)
>>>>    - *A*
>>>>    src/main/java/org/crosswire/jsword/index/lucene/InstalledIndex.java<https://github.com/crosswire/jsword/pull/74/files#diff-3>(67)
>>>>    - *M*
>>>>    src/main/java/org/crosswire/jsword/index/lucene/LuceneIndexManager.java<https://github.com/crosswire/jsword/pull/74/files#diff-4>(53)
>>>>    - *M* src/main/resources/IndexMetadata.properties<https://github.com/crosswire/jsword/pull/74/files#diff-5>(18)
>>>>    - *A* src/main/resources/InstalledIndex.properties<https://github.com/crosswire/jsword/pull/74/files#diff-6>(39)
>>>>    - *A*
>>>>    src/test/java/org/crosswire/jsword/index/lucene/LuceneIndexManagerTest.java<https://github.com/crosswire/jsword/pull/74/files#diff-7>(131)
>>>>
>>>> Patch Links:
>>>>
>>>>    - https://github.com/crosswire/jsword/pull/74.patch
>>>>    - https://github.com/crosswire/jsword/pull/74.diff
>>>>
>>>> --
>>>> Reply to this email directly or view it on GitHub<https://github.com/crosswire/jsword/pull/74>
>>>> .
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> jsword-devel mailing list
>>>> jsword-devel at crosswire.org
>>>> http://www.crosswire.org/mailman/listinfo/jsword-devel
>>>>
>>>>
>>>
>>>
>>> --
>>> Regards,
>>> Sijo
>>> _______________________________________________
>>> jsword-devel mailing list
>>> jsword-devel at crosswire.org
>>> http://www.crosswire.org/mailman/listinfo/jsword-devel
>>>
>>>
>>>
>>> _______________________________________________
>>> jsword-devel mailing list
>>> jsword-devel at crosswire.org
>>> http://www.crosswire.org/mailman/listinfo/jsword-devel
>>>
>>>
>>
>> _______________________________________________
>> jsword-devel mailing list
>> jsword-devel at crosswire.org
>> http://www.crosswire.org/mailman/listinfo/jsword-devel
>>
>>
>
> _______________________________________________
> jsword-devel mailing list
> jsword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/jsword-devel
>
>

-- 
Regards,
Sijo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.crosswire.org/pipermail/jsword-devel/attachments/20140408/15d90b16/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 46501 bytes
Desc: not available
URL: <http://www.crosswire.org/pipermail/jsword-devel/attachments/20140408/15d90b16/attachment-0001.png>