[sword-devel] Catholic versification / inter-versification mappings
Troy A. Griffitts
scribe at crosswire.org
Tue Feb 20 00:52:07 EST 2024
Michael and I obviously differ in opinion on this subject, and he knows
that I love him and so appreciate the work he does at ebible.org.
I would like to propose that our differences are due to our perspective
in light of our primary ministries:
Among many other things, Michael with ebible.org edits and publishes the
World English Bible and its variations and also transforms every freely
distributable Bible from the United Bible Societies' (USB) Digital Bible
Library (DBL) repository to many formats, including SWORD, and for that
we are so grateful and blessed by his efforts.
For his work, he needs to try to fit all these Bibles into one of our
versification schemes (actually maybe he has punted on that and may use
our NRSVA scheme which is one of our largest supersets; I don't know
these days). I can imagine the frustration this must be for him, trying
to automate this conversion as much as possible. He has had much grace
to convert DBL markup to OSIS (a format he doesn't love) and select a
versification, which he sees many of his Bibles don't match exactly. I
am so grateful for his friendship and his partnership in ministry over
many decades.
From my perspective, SWORD is focused on Bible research. In my mind, I
am need to define a finite set of unambiguous segments of Scripture, and
once defined, for each segment:
* show that same segment of Scripture in all Bibles which contain that
segment.
https://crosswire.org/study/parallelstudy.jsp?del=all&add=KJV&add=NA28DBG&add=NASB&add=RST&key=Matt.16.28#cv
* even try to align individual words, if we can, within that segment
(try clicking on any word from the link above).
* find the precise folio in all ancient manuscripts of that segment
and showing images of that segment:
https://crosswire.org/study/fetchdata.jsp?mod=TC&key=Matt.16.28&srcMod=NASB
* find the parallel passages of that segment across the Four Gospels,
using Eusebian Canon Tables, with 2 segments of context before and
after:
https://crosswire.org/study/eusebian.jsp?key=Matt.16.28&context=2#cv
* find and show commentary about that segment:
https://crosswire.org/study/passagestudy.jsp?key=Matt.16.28-28&mod=JFB
* compute and show variation in manuscripts about that segment:
* https://ntvmr.uni-muenster.de/community/vmr/api/collate/?baseText=ECM&indexContent=Matt+16%3A28&extraVerses=0&documentGroupID=-1&ignorePunctuation=true&ignoreSupplied=true&ignoreUnclear=true&ignoreAccents=true&format=graph&preferUser=undefined®UserID=intfadmin
For my daily work, I need tightly defined, unambiguous segments of the
original Text. Michael is daily trying to get a publisher's translation
displaying the way the publisher desires-- with all the publisher's
unique adjustments of the text, and understandably he gets frustrated
that a simple task isn't so simple in SWORD because he needs to segment
and fit each Bible into a versification system with IDs which we can use
to do all our research. I get it. I understand. Simple things should
be simple, and right now they are not.
We have discussed alternatives for a more ePub/PDF like reader's view of
a Bible, where we preserve the imported data file's ordering such that a
reader oriented view could possibly ask for, say 'chapter' chunks and
just display what it gets, and then slice things up into our indices
behind the scenes as we do now. We've had 'reader view' requests for
our lexicon modules, as well, because we order entries to allow fast
binary search lookups, and when a client of the SWORD library asks to
step through a lexicon module, they get an ordered list of entries
instead of a the published order in the original lexicon. We could do
much better preserving original publisher display; even simply to the
point that we support this use case as a first class citizen in the
engine and preserve a ePub or / PDF of the original with each module.
I want to move forward with solutions for each task we each individually
feel called to work toward. I am sorry that I prioritize my tasks I
feel I am called to work toward instead of helping others find solutions
for their ministry. This might be normal for a software engineer, but
not for a good leader.
Serving together,
Troy
On February 19, 2024 18:12:55 MST, Kahunapule Michael Johnson
<kahunapule at eBible.org> wrote:
Dear All,
If I understand Arnaud correctly, I really like his ideas. The BEST
part is that the next time a Bible is submitted for processing with
yet another unique versification (after the changes are
implemented), it doesn't have to be either force-fit into a
versification that doesn't fit or wait for decades for someone to
update the hard-coded versifications in the Sword engine, and for
those to be incorporated into all of the front ends.
I regard the current minimalist versification system to be seriously
in need of an upgrade. It is based on false assumptions (listed by
Troy, no offense intended) that seemed good at the time they were
made. However, with 1404 Bible translations (and counting) is that
(1) 90% success is an over-estimate of how well it works, and (2)
Sword versification is a complete failure for numerous projects
because none of the existing versifications fit, the fall-back
mechanisms fail and result in wrong outputs or crashes in osis2mod,
and nobody is actively fixing the situation.
I have found the following to be true:
The number of versifications needed to represent all Bibles properly
tomorrow is highly likely to be more than the number that works
today. Hard-coding versifications into slowly-changing code that is
only updated in fits and starts is doomed to fail (and already has,
in my not-so-humble opinion).
Verse numbers in a chapter don't always proceed in numerical order.
Several Bible translations move the statement about the motion of
the shadow on Hezekaiah's steps to a more logical place in terms of
discourse, without changing the verse numbers. Indeed, they split
verses into segments and straddle other verses with them.
Chapter and verse "numbers" aren't always pure numbers. Letters get
involved in the Deuterocanon/Apocrypha. Some Bible translators like
to use verse segments (like 6a and 6b) heavily.
Verse bridges (like verse 1-3 with everything from verses 1 through
3 but possibly rearranged and with no other verse markings within
them) are very common.
Mapping any arbitrary versification to any other is NICE, but NOT
NECESSARY. Displaying the text as the translators intended is
NECESSARY. If you can do both, do it. If you cannot, at least
display the versification of the Bible translation as the translator
intended.
I am fully aware of the changes in architecture and code adapting to
the realities I perceive imply. At this point, I'm not sure if
modifying the Sword engine or rewriting it would be easier. Either
way, it is a lot of work.
It is my understanding that JSword is a bit better than Sword in
this regard, in that it doesn't assume fixed versifications.
As far as volunteering for pumpkin holder for versifications, I
nominate Arnaud. (I already bit off more than I can chew by myself.
Sorry.)
On 2/19/24 14:23, Troy A. Griffitts wrote:
>
> Dear all,
>
> These comments are a mix of background, history, and thoughts:
>
> 1) VERSIFICATION (v11n):
>
> Variation between reference systems sucks. Until you get into the
> weeds of the details, it is normal to assume the problems are not
> complex. SWORD tries to implement a simple 90% solution.
>
> SWORD and JSword support defined abstract versification schemes
> with 3 simple dimensions: [bookid : chapterMax][chapterNumber :
> verseMax][verseNumber : verseEntry]
>
> Conceptually we also operate on these assumptions (I've skimmed
> the proposal by Arnaud which differs here, but I haven't given it
> the thought it deserves to comment yet): that book order is
> defined in the v11n system; that chapter and verse numbers are
> numeric and begin at 1 and increase to verseMax. We also allocate
> a special slot '0' for: Module Introduction; Testament
> Introduction; Book Introduction; and Chapter Introduction (e.g.,
> Matt.0.0 can hold an introduction to Matthew).
>
> Those who have been exposed to many Bibles will immediately think
> of places these assumption fall short. But for >90% of our
> Bibles, these assumption hold true, and these assumption make many
> aspects of our work much simpler (abstract parsing of verse lists
> and ranges, bookmark ordering, etc.).
>
> Historically, SWORD previously supported dynamic, per module,
> versification, with a 3 phase lookup:
>
> index file .bks[book number] = book offset in next index;
> index file .cps[book offset + chapter number] = chapter offset in
> next index;
> index file .vss[chapter offset + verse number] = verse offset and
> entry size in data file.
>
> 20 years or so, we made the decision to begin the hard work to
> understand versification systems within Bibles so we could begin
> to map them appropriately. This let us remove the .bks, and .cps
> index files and store that data in versification system
> definitions, leaving only the final .vss index file which gave the
> offsets and entry sizes into the data file.
>
> Caring about versifications was a decision we made. Our previous
> design let any Bible decide how many books, how many chapters, and
> how many verses each chapter contained. This had its merits
> because any new versification could be defined in each module
> without anyone caring what it was. But the drawback was the same:
> any Bible could decide how many books, how many chapters, and how
> many verses without anyone knowing why or what they were.
>
> Some have pushed for dynamic definitions of v11n systems again,
> and I understand why. I am in favor of moving forward with a
> hybrid approach: a set of defined versification systems, which a
> module will still need to choose from, to which it most closely
> adheres, + the ability for that module to specify its variation.
>
> Toward 98%: We have tried to work around the cons of this simple
> design and approach 100% support by accounting for the most common
> types of problems, e.g.
>
> * The engine allows common verse suffixes (e.g. Matt.2.7b);
> * The engine skips verses in a Bible which are not present--
> this allows us to create v11n schemes which are a superset of
> n number of closely related v11n schemes, knowing that the
> engine will skip over the verses that are not present in the
> module; We also have tools which print out missing verses
> which has proven a good QA check for our modules team.
> * When we run across a Bible which adds an odd verse here or
> there or an out of order verse, our workaround has been to
> append these to end of the verse just before where they should
> appear, so the text flows the same as the printed Bible, and
> we include for the reader an inline visual separator and
> marker showing the publisher's verse number.
>
>
> These work arounds get us pretty close to being able to support
> 98% of our Bibles exactly as the publisher wishes, and the
> remaining 2% is supported "well enough" for no complaints by
> publishers. Could we build a system which allowed out of order
> verses, or which allowed any scheme a Bible wished to follow?
> Sure, but the added complexity for various tasks increases quite a
> bit for some of these allowances-- e.g., think index math for book
> chapter verse when we cannot assume numeric sequence; think
> abstract ordering of bookmarks not tied to any specific Bible,
> search results across Bibles, etc.
>
> Our vision with v11n definitions is that they will be a few as
> possible allowing us to map between them most easily; and as many
> as necessary to allow us to represent well enough a published work.
>
> Chris Little previously was our versification pumpkin holder and
> did some amazing work researching all this material. As a
> demonstration of his thorough work and an example of the
> difficulties with v11n, see his work on just the LXX tradition:
>
> https://www.crosswire.org/svn/sword-tools/trunk/versification/lxx_v11ns/
>
> Chris has left our community after many years of volunteering
> massive time and effort.
>
> We haven't had anyone step up who is willing to commit the time
> and effort necessary and who holds our vision (as few as possible,
> as many as necessary).
>
>
> 2) MAPPINGS:
>
> SWORD and JSword support v11n to v11n mappings. Graciously, Костя
> Маслюк worked with us for over a year to discuss the problems and
> implement a versification mapping system which has been included
> in the engine. He also added v11n mappings for systems he was
> interested in supporting. If anyone is interesting in the
> discussions, they can see the archives, e.g.,
>
> https://www.crosswire.org/pipermail/sword-devel/2013-July/040154.html
>
> Historically, we called this topic alternate versification, so if
> you see "av11n" in the archives, you'll be aware.
>
> Registering v11n systems and mappings in the engine is
> straightforward in our versification manager, and as you can see,
> loading these dynamically from a file would be simple to implement:
>
> https://www.crosswire.org/svn/sword/trunk/src/mgr/versificationmgr.cpp
>
> During the development of the mapping infrastructure, our proof of
> concept was to see if we could concisely build a parallel Bible
> HTML display across versification systems, letting a user specify
> any number of Bibles, the first Bible v11n being the primary
> ordering driver:
>
> https://www.crosswire.org/svn/sword/trunk/examples/tasks/parallelBibles.cpp
>
> ____
>
> So, now to the issue with adding another Catholic versification
> system. I would love to continue to delegate ownership of v11n
> decisions! I trusted Chris. He said "no" all the time, and only
> allowed new versification definitions if we really couldn't
> support a set of Bibles using an existing system with our work
> arounds. He spent the time necessary to understand the
> traditions, which published works would use the proposed
> versification, he had excellent skills clearly delineating
> systems-- generally he made well informed decisions from many
> hours of research.
>
> I don't understand the complex details nor have the time to do the
> research for each individual request. My first thought is, where
> is Chris?! Next, my uninformed mind thinks: we have v11n
> definitions "Catholic" and "Catholic2"! Why do we need an
> additional Catholic versification system? Did we do a bad job with
> the first two? Can we not follow our principles and create a
> superset between 2 or more of these? And of course, these are not
> proper responses.
>
> So, if anyone is prayfully willing to take up this pumpkin-- to
> put in the time necessary to research Bible traditions and
> published works, to truly understands both the pros and cons of
> the decisions we've made to go down the path we are on, along with
> our workarounds for the cons, and is willing to live
> wholeheartedly with where we are now, but certainly always open to
> improve, I would love for that person to take ownership of
> versification.
>
> I appreciate the pointers to the Paratext v11ns and mappings,
> maybe we compare where we are now with what they have.
>
> Thank you all for being zealous to improve things. Looking
> forward the conversation to follow,
>
> Troy
>
>
>
> On 1/26/24 06:10, Arnaud Vié wrote:
>
>> Hello everyone,
>>
>> I'm the person Cyrille mentioned, and I just joined the mailing
>> list as I thought I could maybe explain a bit more what I'm
>> trying to do with this new versification.
>>
>> *1. Problem statement*
>>
>> Simply put, my objective is to be able to align verse-by-verse
>> the contents of two bibles that use different versifications.
>> For example :
>> - I open Daniel 3 in a Catholic bible, it has 100 verses because
>> the Prayer of Azariah is included.
>> - I want to compare the translation verse by verse with, for
>> example, the KJVA. This means I want to see the Daniel 3 from
>> KJVA, with all verses from PrAzar included in the corresponding
>> place.
>>
>> There was already logic to perform such mapping in jsword, and I
>> recently included it to support deuterocanonical contents (on the
>> AndBible fork, since that's the only one where I got answers from
>> the maintainer) : https://github.com/AndBible/jsword/pull/13
>>
>> Now, the problem is to be able to do the same with the
>> deuterocanonical additions to the book of Esther, because there
>> are many different "strategies" adopted by different bibles.
>> - Protestant bibles, when they have it, usually have it a
>> separate books (AddEsth in KJVA).
>> - Some catholic bibles have it as additional chapters at the end
>> of Esther, making Esther 16 chapters long : that's the existing
>> "Catholic2" versification, which maps to KJVA easily :
>> https://github.com/AndBible/jsword/blob/develop/src/main/resources/org/crosswire/jsword/versification/Catholic2.properties#L392
>>
>> - Most catholic bibles actually have the additions integrated
>> directly within the original text, using lettered verse numbers
>> (13A, 13B, etc., see here for example :
>> https://www.aelf.org/bible/Est/3 )
>>
>> Currently, these catholic bible with the text integrated use the
>> "Catholic" versification, and ignore all the lettered verses (or
>> include the letters as raw text) : basically, Esther 3.13 with
>> these additions becomes one single very long verse. This makes it
>> impossible to map properly with the AddEsth of KJVA.
>>
>> *2. Proposed solution (Catholic3 versification)*
>>
>> With the proposed Catholic3 versification (except it needs a few
>> adjustments compared to the file proposed by Cyrille), what I'd
>> like to achieve is to give a unique verse number to each of those.
>> For example, Esther 3 goes from 15 to 22 verses, with the OSIS
>> IDs becoming :
>> - Esth.3.13 for verse 13
>> - Esth.3.14 for verse 13A...
>> - Esth.3.20 for verse 13G
>> - Esth.3.21 for verse 14
>> (Basically, the OSIS ID identifies the position of the verse; the
>> actual numbering from the bible can be preserved separately with
>> the OSIS "n" attribute or the "\vp" USFM keyword.)
>>
>> would your use-case be served if canon_catholic.h was
>> modified to increase the verse counts in Esther to 39, 23,
>> 22, 47, 29,
>> 14, 10, 41, 32, 14?
>>
>> Since my objective is to allow mapping verse by verse, you'll
>> understand that I need the verse counts to be aligned with the
>> actual usage. Having the "versification" allow more verses than
>> what's actually used defeats the purpose.
>> In addition, I believe it's a very bad idea to make big changes
>> to already published versifications : the point of versifications
>> is to give a unique ID to a verse. Updating a versification will
>> change all IDs for the verses of already existing bibles that use
>> this versification.
>> I really believe the best solution for the time being is to
>> create a new Catholic3 versification, as originally suggested.
>> I can provide the full definition very soon (though since I'm
>> working with JSword it will be in Java format first), and it
>> should in theory be aligned with Catholic and Catholic2 except
>> for these differences in Esther. (I'll check if there are more
>> differences I missed).
>>
>> *3. Modular versifications*
>>
>> I think at some point it would be nice to have per-book
>> versifications
>> or some other way to deal with bibles that don't follow a
>> "standard"
>> versification
>>
>> Agreed.
>> If everyone is open to the idea, I'd like to work in the next few
>> months on an extension of the OSIS standard to define "modular"
>> versifications, ie. versifications that can be built by composing
>> other versifications and applying a diff.
>> Then each bible could, in its document header, not only reference
>> a standard versification with refSystem, but include its own
>> specific changes and how they map to the standard.
>>
>> Before I spend time on the topic though, is there anyone in
>> particular I should ask to approve the general idea, and who
>> would be interested in reviewing proposals on the topic ?
>>
>> Thanks all and best regards,
>>
>> Arnaud
>>
>> Le jeu. 25 janv. 2024 à 16:35, pinoaffe <pinoaffe at gmail.com> a
>> écrit :
>>
>>
>> Hello,
>>
>> I don't know much about catholic bibles or sword, but just out of
>> curiosity: would your use-case be served if canon_catholic.h was
>> modified to increase the verse counts in Esther to 39, 23,
>> 22, 47, 29,
>> 14, 10, 41, 32, 14? Or would the decreases in verse counts
>> in other
>> chapters of other books also be necessary?
>>
>> And would such a change be acceptable to others?
>>
>> The catholic bibles I've encountered "in the wild" are the dutch
>> Willibrordvertaling of 1975 and the neovulgate. Both of
>> these appear to
>> follow canon_catholic3.h everywhere except in Esther, where
>> they have 16
>> chapters
>>
>> I think at some point it would be nice to have per-book
>> versifications
>> or some other way to deal with bibles that don't follow a
>> "standard"
>> versification, but for now we'll have to make do with what we got
>>
>> Kind regards,
>> pinoaffe
>>
>> Fr Cyrille <fr.cyrille at tiberiade.be> writes:
>> > Hello,
>> > Recently a person interested in the Catholic bible in
>> French told me
>> > about the mapping problems between Catholic versification
>> and the kjva
>> > concerning Esther. So I'm bringing up the question of a new
>> Catholic
>> > versification that could better deal with this kind of
>> problem and at
>> > the same time incorporate some of the errors in current
>> Catholic
>> > versifications. I should mention that two versifications
>> for French
>> > Bibles have been added since my proposal for a new Catholic
>> > versification, which was made long before these
>> versifications were
>> > added, and which was not favorably received at the time. In
>> our case,
>> > the number of Bibles that follow this versification is simply
>> > enormous, since the majority of Catholic Bibles do. I would
>> also add
>> > that the LXX versification is not entirely correct. There
>> are no LXX
>> > today that put 16 chapters to Esther. Ralph's already uses
>> letters to
>> > integrate Greek passages into the text.
>> > In fact, Catholic versification simply follows Ralph's. So
>> I suggest
>> > that we seriously reopen this question with concrete
>> suggestions.
>> > For my part, the suggestion is simple: convert the numbers
>> in the
>> > Greek passages into verses and offset the numbered verses
>> in the same
>> > chapters. I'll copy you on what that might look like.
>> >
>> > Br Cyrille
>> >
>> > [2. text/x-chdr; canon_catholic3.h]...
>> >
>> > _______________________________________________
>> > sword-devel mailing list: sword-devel at crosswire.org
>> > http://crosswire.org/mailman/listinfo/sword-devel
>> > Instructions to unsubscribe/change your settings at above page
>>
>>
>> _______________________________________________
>> sword-devel mailing list:sword-devel at crosswire.org
>> http://crosswire.org/mailman/listinfo/sword-devel
>> Instructions to unsubscribe/change your settings at above page
>
> _______________________________________________
> sword-devel mailing list:sword-devel at crosswire.org
> http://crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://crosswire.org/pipermail/sword-devel/attachments/20240219/ff2f0ab8/attachment-0001.htm>
More information about the sword-devel
mailing list