[jsword-devel] [sword-devel] Method to find if BibleBook is contained in a Book
DM Smith
dmsmith at crosswire.org
Fri Mar 28 13:50:20 MST 2014
It will be performant with Bibles.
JSword is stable at the tip. I've just checked in the bug fix that Chris supplied.
This change will be stable.
-- DM Smith
On Mar 28, 2014, at 4:34 PM, Martin Denham <mjdenham at gmail.com> wrote:
> I was only thinking of using it with SwordBook/AbstractPassageBook but if it is not performant then maybe it is not worth continuing and we should look at Scope. I thought that it was already being calculated in ZVerseBackend.contains() using the idxRaf.
>
> btw is it safe to get the tip of JSword yet?
>
> Martin
>
>
> On 28 March 2014 20:19, DM Smith <dmsmith at crosswire.org> wrote:
> I think it would be good to support Scope formally, even if it never makes it into SWORD. As a different issue, we'll be changing JSword to keep a module's conf pristine and the things that we write to it, will be put into a side-car conf. This will be the perfect place for us to compute the value once for all time per module.
>
> The getRawTextLength is not as easy as I'd like. It's mostly done. A bit more to do. For a couple of module types, both compressed, it is not performant. It merely calls getRawText and then length. The problem is that one has to uncompress the text to see how long it is.
>
> -- DM
>
> On Mar 28, 2014, at 3:31 PM, Martin Denham <mjdenham at gmail.com> wrote:
>
>> An alternative method might be to use the Scope value which IBT have placed in the .conf file, but I can't seem to get access to it via JSword.
>>
>> This is printed:
>> WARNING: Extra entry in kaz of Scope
>>
>> And in ConfigEntryTable:
>> log.warn("Extra entry in {} of {}", internal, configEntry.getName());
>> extra.put(key, configEntry);
>>
>> But I can't see any way to get the value from the extra map? Is it possible - I am a bit confused by the initialisation and retrieval of metadata and properties in JSword.
>>
>> Example scopes from IBT modules
>>
>> Scope for kaz:
>> Scope=Gen-Josh.24.33 Judg-2Chr Ezra-Neh Esth-Ps.150 Prov.0-Prov.4.27 Prov.5-Prov.13.25 Prov.14-Prov.18.24 Prov.19-Song Isa-Lam Ezek-Dan.3.33 Dan.4-Dan.12 Hos-Mal Matt-Rev
>>
>> Scope for kylsc:
>> Scope=Matt-Rev
>>
>> I don't know if the strings used are compatible with PassageKeyFactory but if we only look at the start and end of the scope we may be able to deduce all that is required because I think IBT are the only people who use scope.
>>
>> Martin
>>
>>
>>
>> On 28 March 2014 14:12, DM Smith <dmsmith at crosswire.org> wrote:
>> I'll add the method SwordBook.getRawTextLength(Key key), or something like it. -- DM
>>
>> On Mar 26, 2014, at 6:47 PM, Martin Denham <mjdenham at gmail.com> wrote:
>>
>>> Given the above explanations and that many users have already downloaded such modules I have experimented with a work-around by adding some extra logic to And Bible to specifically cater for the IBT Synodal modules. I did this by making the assumption that all the empty verses start with: "<chapter eID=" which appears true and unique. It is a bit of a hack but it almost worked.
>>>
>>> The only problem is that after adding the extra getRawText checks it takes too long, even on my Nexus 4, to load the book list for IBT modules. However, a simpler way to avoid the getRawText calls would be to add a
>>> public int SwordBook.getRawTextLength(Key key)
>>> which would be identical code to contains(Key key) (->ZVerseBackend.contains) but return verse length instead of a boolean (contains() calculates verse length to determine if a verse exists). What do you think? This would help because IBT empty verse stubs are very short and so normally the getRawText would not be required as part of the elaborated contains() check in And Bible.
>>>
>>> Note:
>>> I have discovered that this problem does not just affect deuterocanonical books in IBT Synodal modules, it also affects OT books in IBT NT-only modules e.g. KYLSC, which return text like "<chapter eID="gen4" osisID="Gen.1"/>".
>>>
>>> Martin
>>>
>>>
>>> On 26 March 2014 14:49, DM Smith <dmsmith at crosswire.org> wrote:
>>> John,
>>>
>>> Putting this up on sword-devel, since that is a more appropriate location for the discussion to continue. This is really not about JSword, but rather about module making.
>>>
>>> The nature of osis2mod is to retain all markup except <verse> and </verse> (or their equivalent milestoned version.) This means that the markup for a chapter is put in the module's storage for that chapter and noted in the index. In the case of the chapter that is given below, it is split into 2 parts, Verse 0 and Verse 1.
>>> Verse 0 will get the preamble of the chapter:
>>> <chapter osisID="EpJer.1">
>>> Verse 1 will get:
>>> </chapter>
>>> (These will have been transformed into their milestoned versions.)
>>>
>>> Also, verse 2 to 72 will be "linked" to verse 1, meaning that in the index they are given the same location as verse 1.
>>>
>>> So, verse 0 has chapter start content and verse 1 to 72 have chapter end content.
>>>
>>> Also, osis2mod does not complain if a verse is missing. Never has, never will. It does "complain" of a verse being present that is not in the versification. Always has, always will.
>>>
>>> That emptyvss indicates that all verses are present means exactly that: All verses are present. This is not good if the module is in fact incomplete.
>>>
>>> That JSword indicates that these "empty" verses are present means that they have non-zero length in the module.
>>>
>>> JSword is graceful in handling this. It determines that the module has content for the verse by examining the index. What Martin is trying to do is find out which books, chapters and verses should be displayed to users in pick lists. The only way this can be done at this time, by either SWORD or JSword with the module in question, is to render each verse and determine that it renders nothing. This is far too expensive an operation to consider.
>>>
>>> The only way to efficiently determine scope is to examine the index for each verse and see if the length is 0. The Scope entry in the conf has been ruled out. It would have been computed using the reverse logic of emptyvss. Go through the v11n from first verse to last and rather than noting what is missing, note what is present.
>>>
>>> Today, most of our frontends display pick lists based on the v11n not on the module content. It has long been confusing to end users of modules that don't contain verses in the v11n.
>>>
>>> In my view, this is a module problem. It is far easier and faster to rebuild and redistribute a module. We can tell a user to upgrade to the most recent version of a module far easier than making and releasing a code change and having them get a new version of the program. When the change is a work-around for something that shouldn't be in module, I think we should avoid that. For example, the NET Bible has some bugs that should be fixed. But instead we have some special code that is essentially: if module is NET then fix such-and-so when it occurs.
>>>
>>> Together in His Service,
>>> DM Smith
>>>
>>>
>>>
>>> On Mar 25, 2014, at 11:43 PM, John Austin <gpl.programs.info at gmail.com> wrote:
>>>
>>>> There has been a lot of discussion about how missing material in a v11n should be treated (the discussion of the meaning and use of Scope was part of that). Tools such as osis2mod generated warnings whenever OSIS files lacked any part of the chosen v11n. The Scope conf param was, for a time at least, the recommended method of describing what part of a v11n was covered by a module. For these reasons, many existing modules (IBT alone has at least 26 such modules) are currently encoded so as to encompass the entire v11n, returning empty-string verse content for all verses in the v11n that are not included in the module, and using the .conf Scope param to define exactly what is present in the module.
>>>>
>>>> So even though current module making best practice may be different, it would be good for JSword to be graceful with modules that are encoded somewhat differently if at all possible, at least for a time. There are many modules out there, old and new, which don't contain the complete v11n, so determining book coverage is important.
>>>>
>>>> -John
>>>>
>>>>
>>>>
>>>> On 03/25/2014 08:19 PM, DM Smith wrote:
>>>>> Those verses exist since they are defined in the OSIS input file to
>>>>> osis2mod. Osis2mod retains everything in its input. This is a well
>>>>> documented behavior of osis2mod.
>>>>>
>>>>> The end chapter markup will be put in the last verse that is in the
>>>>> chapter, which might be verse 0.
>>>>>
>>>>> They should use xslt to strip empty verses, chapters and books out of
>>>>> their file into an intermediate file and give that as input to osis2mod.
>>>>>
>>>>> Alternatively they can use <!-- ... --> to comment out huge swaths of
>>>>> the input file.
>>>>>
>>>>>
>>>>> -- DM
>>>>>
>>>>> On Mar 25, 2014, at 7:48 AM, Martin Denham <mjdenham at gmail.com
>>>>> <mailto:mjdenham at gmail.com>> wrote:
>>>>>
>>>>>> IBT have just passed me more information regarding their handling of
>>>>>> empty verses to help clarify if this is an IBT module issue or not.
>>>>>> The following is an extract from IBT's e-mail:
>>>>>>
>>>>>> Here are examples of how IBT's OSIS source defines empty verses in
>>>>>> the markup:
>>>>>>
>>>>>> Empty book (Epistle of Jeremiah):
>>>>>> <div type="x-Synodal-non-canonical"__><div type="book"
>>>>>> osisID="EpJer"><chapter osisID="EpJer.1"><verse sID="EpJer.1.1-72"
>>>>>> osisID="EpJer.1.1 EpJer.1.2 EpJer.1.3 EpJer.1.4 EpJer.1.5
>>>>>> EpJer.1.6 EpJer.1.7 EpJer.1.8 EpJer.1.9 EpJer.1.10 EpJer.1.11
>>>>>> EpJer.1.12 EpJer.1.13 EpJer.1.14 EpJer.1.15 EpJer.1.16 EpJer.1.17
>>>>>> EpJer.1.18 EpJer.1.19 EpJer.1.20 EpJer.1.21 EpJer.1.22 EpJer.1.23
>>>>>> EpJer.1.24 EpJer.1.25 EpJer.1.26 EpJer.1.27 EpJer.1.28 EpJer.1.29
>>>>>> EpJer.1.30 EpJer.1.31 EpJer.1.32 EpJer.1.33 EpJer.1.34 EpJer.1.35
>>>>>> EpJer.1.36 EpJer.1.37 EpJer.1.38 EpJer.1.39 EpJer.1.40 EpJer.1.41
>>>>>> EpJer.1.42 EpJer.1.43 EpJer.1.44 EpJer.1.45 EpJer.1.46 EpJer.1.47
>>>>>> EpJer.1.48 EpJer.1.49 EpJer.1.50 EpJer.1.51 EpJer.1.52 EpJer.1.53
>>>>>> EpJer.1.54 EpJer.1.55 EpJer.1.56 EpJer.1.57 EpJer.1.58 EpJer.1.59
>>>>>> EpJer.1.60 EpJer.1.61 EpJer.1.62 EpJer.1.63 EpJer.1.64 EpJer.1.65
>>>>>> EpJer.1.66 EpJer.1.67 EpJer.1.68 EpJer.1.69 EpJer.1.70 EpJer.1.71
>>>>>> EpJer.1.72"/><verse eID="EpJer.1.1-72"/></chapter>__</div></div>
>>>>>>
>>>>>> I'm not sure how osis2mod handles all this when importing to the
>>>>>> module, but it works perfectly without warnings or errors. Also,
>>>>>> when the resulting module is passed to the "emptyvss" tool, it
>>>>>> passes this test without warnings.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 25 March 2014 11:38, Martin Denham <mjdenham at gmail.com
>>>>>> <mailto:mjdenham at gmail.com>> wrote:
>>>>>>
>>>>>> I am having problems getting a list of BibleBooks contained in
>>>>>> some AV modules which we know do not contain certain books. I
>>>>>> can't work out if the problem is with JSword, the modules, or
>>>>>> osis2mod.
>>>>>>
>>>>>> There are 2 related problems I can see:
>>>>>>
>>>>>> 1. book.contains(nonExistingVerse) returns TRUE
>>>>>> 2. book.getRawText(nonExistingVerse) returns <chapter end tag>
>>>>>>
>>>>>> Here is a simple test to show the problem using KAZ which has
>>>>>> Synodal v11n but does not contain any deuterocanonical books:
>>>>>>
>>>>>> SwordBook kaz = (SwordBook)Books.installed().getBook("KAZ");
>>>>>> Verse esd11Verse = new Verse(kaz.getVersification(),
>>>>>> BibleBook.ESD1, 1, 1);
>>>>>> System.out.println(kaz.contains(esd11Verse));// prints: *true*
>>>>>> System.out.println(kaz.getRawText(esd11Verse));// prints:
>>>>>> *<chapter eID="gen7" osisID="1Esd.1"/>*
>>>>>> Verse esd12Verse = new Verse(kaz.getVersification(),
>>>>>> BibleBook.ESD1, 1, 2);
>>>>>> System.out.println(kaz.contains(esd12Verse));// prints: *true*
>>>>>> System.out.println(kaz.getRawText(esd12Verse));// prints:
>>>>>> *<chapter eID="gen7" osisID="1Esd.1"/>*
>>>>>>
>>>>>> So how does "<chapter eID="gen7" osisID="1Esd.1"/>" get into verse
>>>>>> content unexpectedly?
>>>>>>
>>>>>> It seems to me like it could be either:
>>>>>>
>>>>>> 1. a module problem; but IBT say they do not add empty verse slots
>>>>>> 2. Sword osis2mod issue
>>>>>> 3. JSword issue: why is JSword returning a chapter end tag
>>>>>> instead of verse content
>>>>>>
>>>>>> Any ideas what might cause this problem?
>>>>>>
>>>>>> Thanks
>>>>>> Martin
>>>>>>
>>>>>>
>>>>>> On 11 March 2014 12:15, DM Smith <dmsmith at crosswire.org
>>>>>> <mailto:dmsmith at crosswire.org>> wrote:
>>>>>>
>>>>>> We haven't pushed this down into JSword. So far it is the
>>>>>> responsibility of the front-end. Chris B has made it efficient
>>>>>> to ask a Book whether it contains a Verse.
>>>>>>
>>>>>> Essentially, when it comes to asking a module if it has
>>>>>> meaningful content, you want containsAny(Key verses, boolean
>>>>>> includeIntros) and containsAny(Key verses) { return
>>>>>> containsAny(verses, false); }
>>>>>>
>>>>>> I think it should ignore verse 0 by default. If it doesn't
>>>>>> have verse content, then does the content really mean something?
>>>>>>
>>>>>> As you have noted contains(Key) is confusing. There are a few
>>>>>> places where it means containsAny. Usually it means
>>>>>> containAll. The name, contains, was chosen early as we derived
>>>>>> from a container class where the argument was an element of
>>>>>> the container. That is, contains is supposed to mean
>>>>>> isMemberOf. Later we changed the inheritance as it wasn't an
>>>>>> "is a" relationship.
>>>>>>
>>>>>> But we need to be careful of not introducing more confusion.
>>>>>>
>>>>>> By the way, the list serve was holding mail for a few days.
>>>>>>
>>>>>> In Him,
>>>>>> DM
>>>>>>
>>>>>> On Mar 8, 2014, at 5:26 PM, Martin Denham <mjdenham at gmail.com
>>>>>> <mailto:mjdenham at gmail.com>> wrote:
>>>>>>
>>>>>> > Is there an efficient way to find if a BibleBook is
>>>>>> contained in a Book (Bible or commentary) using JSword?
>>>>>> >
>>>>>> > I recall this subject being discussed but can't recall the
>>>>>> outcome.
>>>>>> >
>>>>>> > Thanks
>>>>>> > Martin
>>>
>>> _______________________________________________
>>> sword-devel mailing list: sword-devel at crosswire.org
>>> http://www.crosswire.org/mailman/listinfo/sword-devel
>>> Instructions to unsubscribe/change your settings at above page
>>>
>>> _______________________________________________
>>> jsword-devel mailing list
>>> jsword-devel at crosswire.org
>>> http://www.crosswire.org/mailman/listinfo/jsword-devel
>>
>>
>> _______________________________________________
>> jsword-devel mailing list
>> jsword-devel at crosswire.org
>> http://www.crosswire.org/mailman/listinfo/jsword-devel
>>
>>
>> _______________________________________________
>> jsword-devel mailing list
>> jsword-devel at crosswire.org
>> http://www.crosswire.org/mailman/listinfo/jsword-devel
>
>
> _______________________________________________
> jsword-devel mailing list
> jsword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/jsword-devel
>
>
> _______________________________________________
> jsword-devel mailing list
> jsword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/jsword-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.crosswire.org/pipermail/jsword-devel/attachments/20140328/fea6fda5/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4145 bytes
Desc: not available
URL: <http://www.crosswire.org/pipermail/jsword-devel/attachments/20140328/fea6fda5/attachment-0001.p7s>
More information about the jsword-devel
mailing list