[sword-devel] Versification Mapping

Fri May 8 23:44:13 MST 2020

Thanks, Troy!

These numbers below look excellent! I will look into using the 
VerseKey-based mapping approach for Ezra Project. However, it's going to 
be a significant refactoring effort, so I'm probably not going to get 
back with implementation-based feedback any time soon. I just created an 
Issue for Ezra Project 
<https://github.com/tobias-klein/ezra-project/issues/57> and I think 
this will be part of 0.14.0 (The release after the one I'm working on 
currently).

Regarding a shared/common bookmarking and tagging concept the question 
is whether we only talk about a shared "export / import format" or 
whether we talk about a complete persistence backend that could be 
shared. Currently I'm storing my user data in a SQLite database and it 
works well (including automated migrations in case of db schema changes 
that are applied with new versions of the application). I could imagine 
that even the discussion about the storage backend could take a while in 
terms of alignment. I realize that developers have quite different taste 
when it comes to stuff like this. Also the question would be why this 
should be limited to tags. In general it also applies to any other user 
data like notes or user markup for the bible text.

Something more realistic than a shared storage backend would be a shared 
export / import format. This I think may be worthwhile to pursue. We 
could simply agree on a certain json or xml format for this.
@all: Who would be interested in this in general? We could start with 
tags / topical verse lists.

The data model in Ezra Project is currently rather simple. It's just a 
flat list of tags where each tag can be assigned to any verse. In my 
database I currently have three tables to support this. Tags (the actual 
list of topics/keywords), VerseTags (the join/assignment table) and 
VerseReferences (the abstract/module-independent objects tags get 
assigned to).

A tree introduces additional challenges in the user interface. I've been 
thinking about introducing a tree structure for the tags, not anytime 
soon though ... it's a rather big effort from a user interface perspective.

I'm certainly curious about further input to this discussion.

Tobias

On 5/7/20 6:54 PM, Troy A. Griffitts wrote:
>
> Dear Tobias,
>
> I would recommend KJVA, so you can get the Apocrypha.  I believe this 
> is the common base Костя uses to convert between systems.  Костя has 
> done a great job at optimization.  I've updated the verseconvert.cpp 
> example to allow ranges.  Here is a timing for the entire book of Psalms:
>
> [scribe at localhost classes]$ time ./verseconvert Ps Wycliffe 
> FreGeneve1669 > /dev/null
>
> real    0m0.195s
> user    0m0.162s
> sys    0m0.033s
>
> My guess is that most of that time is simply with SWORD engine 
> initialization, discovering and opening my library.
>
> Converting all verses in both Old and New Testament:
>
> [scribe at localhost classes]$ time ./verseconvert Gen-Rev Wycliffe 
> FreGeneve1669 > /dev/null
>
> real    0m0.645s
> user    0m0.601s
> sys    0m0.042s
>
> Hope this is helpful.  Looking forward to your work.  We've had 
> discussions over the many years of our project about sharing 
> bookmarking and similar data between apps, but we all had different 
> ideas of what we'd like.  BibleCS (our Windows classic app) 
> concentrates on allowing verse groups and trees of groups so users can 
> create trees of topic.  Other apps extend this to allow comments and 
> notes at each reference, including a specific module name with the 
> reference.  We've all had different data formats for our bookmarking 
> materials.  Since your app centers on tagging (which I would place in 
> the same category as bookmarking), I would love to hear what you come 
> up with and if it encompasses everything which all of our frontends 
> desire to provide their users.  Maybe we can make another go at a 
> shared "bookmarking" or "tagging" concept and push that concept back 
> into the engine so all app can share and take advantage of your 
> functionality, if you'd be willing.
>
> Troy
>
>
> On 5/7/20 9:07 AM, Tobias Klein wrote:
>> Thanks Troy!
>>
>> Besides comparing verses in different translations my primary use 
>> case is the following:
>> Establish „unique verse reference“ objects that I can use to assign 
>> tags independent of the specific translation.
>>
>> Currently I’m doing this by determining the „absolute verse number“ 
>> for the respective verse I’m tagging.
>> Then at the point when the „verse reference object“ (a database 
>> entry) is created, the absolute verse number both for English and 
>> Hebrew versification are determined based on my mapping tables.
>>
>> Then, when looking up tags for the currently selected verses I can 
>> map the correct „verse reference object“ to each verse in the 
>> currently selected translation (based on the respective absolute 
>> verse number).
>>
>> If I migrate to the Sword VerseKey approach I would change my concept 
>> in the following way:
>>
>> 1) Use KJV verse references (chapter + verse) as unique identifiers 
>> for all „abstract verse reference“ objects. (Does KJV make sense as a 
>> „ reference versification“)
>> 2) When opening a translation that does not follow the KJV theme - 
>> map all verse references to the KJV variants and use the KJV verse 
>> references (chapter + verse) as a key to find the correct „verse 
>> reference“ objects in my database.
>>
>> What’s the performance of the SWORD VerseKey based mapping function 
>> you described below?
>>
>> How would this work if I need to perform a mapping operation on all 
>> verses of a large book like Psalms? (2461 verses).
>> Would this operation come back in less than a second?
>>
>> If not I would have to think about some caching mechanism in my verse 
>> reference objects. Currently I’m caching the absolute verse numbers 
>> for English and Hebrew for all tagged verse references.
>> With the new concept I could cache the actual chapter/verse 
>> references for all known versification systems.
>>
>> Thanks for your help on this!
>>
>> Best regards,
>> Tobias
>>
>>> Am 06.05.2020 um 18:47 schrieb Troy A. Griffitts 
>>> <scribe at crosswire.org <mailto:scribe at crosswire.org>>:
>>>
>>> Yes, as Peter has pointed out, SWORD includes a facility for 
>>> mapping, graciously contributed by Костя 
>>> Маслюк<kostyamaslyuk at gmail.com>and should "just work" when setting a 
>>> key from one module to another, e.g, 
>>> kjv->setKey(wycliffe->getKey()). It's not quite that simple, because 
>>> there isn't always a 1:1 mapping, so if you want full support, 
>>> you'll have to see if a bound is set on the receiving module's key.
>>>
>>> The mapping data, as with everything, is not exhaustive, but we'd 
>>> certainly like to extend it to meet cases which you run into which 
>>> aren't yet supported.
>>> You can see it taken advantage of in example 
>>> sword/examples/tasks/parallelbibles.cpp, but I've just added a 
>>> concise example which shows how to use it:
>>>
>>> http://crosswire.org/svn/sword/trunk/examples/classes/verseconvert.cpp
>>>
>>> Which outputs, e.g.
>>> ./verseconvert Ps.43.22 Wycliffe FreGeneve1669
>>>
>>> Psalms 43:22 (Wycliffe) => Psalms 44:21-Psalms 44:22 (FreGeneve1669)
>>>
>>> Hope this helps,
>>> Troy
>>>
>>>
>>>
>>>
>>>
>>> On 5/6/20 7:07 AM,refdoc at gmx.netwrote:
>>>> I think transparent mapping has been for a while now included in 
>>>> the library. I am not sure how to make it work, but I do think it 
>>>> is there and functioning.
>>>>
>>>> Peter
>>>>
>>>> Sent from my mobile. Please forgive shortness, typos and weird 
>>>> autocorrects.
>>>>
>>>>
>>>> -------- Original Message --------
>>>> Subject: Re: [sword-devel] Versification Mapping
>>>> From: Jamie
>>>> To: 'SWORD Developers' Collaboration Forum'
>>>> CC:
>>>>
>>>>
>>>>     Hi Tobias,
>>>>     Not sure that this exactly answers your question, but just in
>>>>     case it’s relevant, Tyndale House have various public domain
>>>>     information available, including material on alternative
>>>>     versification schemes.  The reversification material gives
>>>>     details of how to map LXX, MT and Vulgate schemes on to NRSVA
>>>>     (and also addresses some other schemes which are perhaps less
>>>>     frequently encountered).  It also caters for common variants
>>>>     which basically follow one of these schemes, but which have
>>>>     certain verses split up into subverses.  You can find the data
>>>>     at :-
>>>>     https://github.com/tyndale/STEPBible-Data/blob/master/TVTMS%20-%20Tyndale%20Versification%20Traditions%20with%20Methodology%20for%20Standardisation%20for%20Eng%2BHeb%2BLat%2BGrk%2BOthers%20-%20TyndaleHouse.com%20STEPBible.org%20CC%20BY-NC.txt
>>>>     If you do want to make use of it, I’d be very happy to try to
>>>>     answer any questions.
>>>>     Regards,
>>>>     ARA “Jamie” Jamieson
>>>>     *From:*Tobias Klein [mailto:contact at tklein.info]
>>>>     *Sent:*05 May 2020 21:19
>>>>     *To:*SWORD Developers' Collaboration
>>>>     Forum<sword-devel at crosswire.org>
>>>>     *Subject:*[sword-devel] Versification Mapping
>>>>
>>>>     Hi,
>>>>
>>>>     I would like to ask a question that I was planning to ask for a
>>>>     while already ...
>>>>     What's the recommended solution of mapping different
>>>>     versification systems?
>>>>
>>>>     And what working implementations for this are already out there?
>>>>
>>>>     I realize that my understanding of versifications has been a
>>>>     bit limited and that's visible in Ezra Project's implementation
>>>>     of the mapping. I am currently only differentiating between two
>>>>     versification systems, namely the English versification (used
>>>>     in most/all (?) English translations) and the Hebrew
>>>>     versification (used in most modern German translations).
>>>>
>>>>     It's been a few years since I looked into this and I think this
>>>>     has been my source (SBL Handbook of Style)
>>>>     https://books.google.de/books?id=M_upBwAAQBAJ&pg=PA265&lpg=PA265&dq=appendix+english/hebrew/greek+versification&source=bl&ots=CXVR0J6YrI&sig=ACfU3U3hEIPgNxmmUQW1kZJaRAtHl78L-g&hl=de&sa=X&ved=2ahUKEwilyoPUwp3pAhUrzqYKHVk4BtIQ6AEwAXoECAYQAQ#v=onepage&q=appendix%20english%2Fhebrew%2Fgreek%20versification&f=false
>>>>
>>>>     My current approach in Ezra Project to map between English and
>>>>     Hebrew versification is the following:
>>>>
>>>>       * I use "absolute verse numbers" in each book.
>>>>       * I have mapping tables that basically define offsets for the
>>>>         "absolute verse numbers" (see implementationhere
>>>>         <https://github.com/tobias-klein/ezra-project/blob/master/models/versereference.js#L177>).
>>>>       * The versification (currently only English or Hebrew) of the
>>>>         respective translation is detected based on some simple
>>>>         dynamic tests when opening it.
>>>>       * I have functions to convert between one and the other
>>>>         "absolute verse numbers" based on the mapping.
>>>>       * Verse Reference objects are stored both with the English
>>>>         and Hebrew absolute verse numbers and these objects are
>>>>         used for assigning tags, notes, etc.
>>>>
>>>>     This works fairly well when using English translations and
>>>>     German translations. The result is for example that tags that
>>>>     were assigned to verses of an English translation still show up
>>>>     correctly for the verses in a German translation. This is
>>>>     particularly visible in Psalms.
>>>>
>>>>     How flawed is my current approach described above?
>>>>     How do other frontends do it?
>>>>     Have there been plans to somehow integrate some sort of mapping
>>>>     functionality into the SWORD engine?
>>>>
>>>>     Best regards,
>>>>     Tobias
>>>>
>>>>
>>>> _______________________________________________ sword-devel mailing 
>>>> list:sword-devel at crosswire.orghttp://www.crosswire.org/mailman/listinfo/sword-develInstructions 
>>>> to unsubscribe/change your settings at above page
>>> _______________________________________________
>>> sword-devel mailing list:sword-devel at crosswire.org 
>>> <mailto:sword-devel at crosswire.org>
>>> http://www.crosswire.org/mailman/listinfo/sword-devel
>>> Instructions to unsubscribe/change your settings at above page
>>
>>
>> _______________________________________________
>> sword-devel mailing list:sword-devel at crosswire.org
>> http://www.crosswire.org/mailman/listinfo/sword-devel
>> Instructions to unsubscribe/change your settings at above page
>
> _______________________________________________
> sword-devel mailing list: sword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.crosswire.org/pipermail/sword-devel/attachments/20200509/9508690d/attachment-0001.html>