<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br><div><div>On Apr 29, 2009, at 4:06 AM, Tonny Kohar wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div>Hi,<br><br>On Mon, Apr 27, 2009 at 4:42 PM, Tonny Kohar <<a href="mailto:tonny.kohar@gmail.com">tonny.kohar@gmail.com</a>> wrote:<br><blockquote type="cite"><br></blockquote><blockquote type="cite">My initial finding seem there is no need for API change, what it needs<br></blockquote><blockquote type="cite">is simple a new package eg: o.c.jsword.index.highlight<br></blockquote><blockquote type="cite">and inside that package there is a static/factory/builder Highlight<br></blockquote><blockquote type="cite">class which accept either (raw text, OSIS xml, or html output).<br></blockquote><blockquote type="cite">....<br></blockquote><br>After tried that approach for few days, sorry that one does not works<br>:), since it is not based on the original index.<br><br>So I tried another approach by patching the LuceneIndex and trying to<br>store (in array/map) the Doc.FieldContent which matched the query, to<br>avoid changing the index structure (eg: adding positional field). And<br>applying that into the osis xml text, this approach does not work<br>either (or have difficulty mapping it back) if the query is phrase<br>because of the osis xml structure eg: a word can be surrounded with<br>lemma/strong tag, but this works for fuzzy search (non phrase)<br><br>So I guess I will another approach in the next few days, if still<br>couldn't get it right then I guess I will put aside that search<br>highlight due to the complexity involved and revisit later.</div></blockquote><div><br></div>Tonny,</div><div><br></div><div>I am excited that you are working on this!</div><div><br></div><div>I don't have a problem with changing the structure of the lucene index to include information that is useful. At some point, the index will change.</div><div><br></div><div>We have the start of a mechanism to track the change so that the UI can notify a user properly that a index rebuild is advisable. Here are some scenarios:</div><div>1) More features are available in the engine. E.g. a new field can be searched. Highlighting can be done. ...</div><div>2) The index has changed (like in the case of adding position/offset)</div><div>3) The underlying engine, lucene, has changed.</div><div><br></div><div>The class o.c.j.index.lucene.IndexMetadata is used to track the current implementation levels. It is used in only one place in JSword.</div><div><br></div><div>The class needs to be moved to o.c.j.index and the method <span class="Apple-style-span" style="font-family: Monaco; font-size: 11px; ">getLuceneVersion needs to be changed to getImplementationVersion and a getImplemenationName needs to be added, in case we need to change engine.</span></div><div><font class="Apple-style-span" face="Monaco" size="3"><span class="Apple-style-span" style="font-size: 11px;"><br></span></font></div><div><font class="Apple-style-span" face="Monaco" size="3"><span class="Apple-style-span" style="font-size: 11px;">When an index is created, this metadata needs to be stored with the index.</span></font></div><div><font class="Apple-style-span" face="Monaco" size="3"><span class="Apple-style-span" style="font-size: 11px;"><br></span></font></div><div><br></div><div><font class="Apple-style-span" face="Monaco" size="3"><span class="Apple-style-span" style="font-size: 11px;">If the implementation version has changed, it may need to be rebuilt. The number is 3 part: Major.minor.point. If the major has changed, then it needs to be rebuilt. If the minor has changed, it'd probably be best to rebuild. If the point has changed, it doesn't need to be rebuilt.</span></font></div><div><font class="Apple-style-span" face="Monaco" size="3"><span class="Apple-style-span" style="font-size: 11px;"><br></span></font></div><div><font class="Apple-style-span" face="Monaco" size="3"><span class="Apple-style-span" style="font-size: 11px;"><span class="Apple-style-span" style="font-family: Helvetica; font-size: 12px; "><div><font class="Apple-style-span" face="Monaco" size="3"><span class="Apple-style-span" style="font-size: 11px; ">When an index is used, if the index version has changed, then the index needs to be rebuilt. It'd probably be good to go to the major.minor.point notation for our releases.</span></font></div><div><font class="Apple-style-span" face="Monaco" size="3"><span class="Apple-style-span" style="font-size: 11px;"><br></span></font></div><div><font class="Apple-style-span" face="Monaco" size="3"><span class="Apple-style-span" style="font-size: 11px;">Right now, the index version is a simple incrementing number, starting at 1.1, but it could change easily, give that there is one use of it.</span></font></div><div><font class="Apple-style-span" face="Monaco" size="3"><span class="Apple-style-span" style="font-size: 11px;"><br></span></font></div><div><font class="Apple-style-span" face="Monaco" size="3"><span class="Apple-style-span" style="font-size: 11px;">IndexManager should be changed to ask for a recommendation on whether to rebuild the index. From the above, it should be a multi-level answer.</span></font></div><div><font class="Apple-style-span" face="Monaco" size="3"><span class="Apple-style-span" style="font-size: 11px;"><br></span></font></div><div><font class="Apple-style-span" face="Monaco" size="3"><span class="Apple-style-span" style="font-size: 11px;">Another thought: it might be nice to have a feature list with index version numbers. For example, the one place that uses the class is to check to see if the index version number is >= is for a feature that was added at a particular point. So if we add a language analyzer, say Indonesian, that would be present at say, 1.3, but not earlier. So if an Indonesian Bible had been indexed before 1.3, it can now be indexed better.</span></font></div><div><font class="Apple-style-span" face="Monaco" size="3"><span class="Apple-style-span" style="font-size: 11px;"><br></span></font></div><div><font class="Apple-style-span" face="Monaco" size="3"><span class="Apple-style-span" style="font-size: 11px;">In Him,</span></font></div><div><font class="Apple-style-span" face="Monaco" size="3"><span class="Apple-style-span" style="font-size: 11px;"><span class="Apple-tab-span" style="white-space:pre"> </span>DM<br></span></font></div><div><font class="Apple-style-span" face="Monaco" size="3"><span class="Apple-style-span" style="font-size: 11px;"><br></span></font></div><div><font class="Apple-style-span" face="Monaco" size="3"><span class="Apple-style-span" style="font-size: 11px;"><br></span></font></div><div><font class="Apple-style-span" face="Monaco" size="3"><span class="Apple-style-span" style="font-size: 11px;"><br></span></font></div><div><font class="Apple-style-span" face="Monaco" size="3"><span class="Apple-style-span" style="font-size: 11px;"><br></span></font></div></span></span></font></div><br></body></html>