[tyndale-devel] Strongs Tagging for ESV

David Instone-Brewer davidinstonebrewer at gmail.com
Sun Mar 20 06:24:25 MST 2011 

I think I found the answer to my question in the credits in the KJV 
module, which thanks "the Bible Foundation for keying the 
Hebrew/English data and [CrossWire's] Project KJV2003 for working 
toward the completion of synchronizing the English phrases to the 
Stephanas Textus Receptus, and to Dr. Maurice Robinson for providing 
the base Greek text with Strong's and Morphology"

David IB
=================

Troy, is there more than one KJV tagging system?
BIbleWorks says their Strongs tagging comes from Online.

David IB


At 00:15 20/03/2011, David Instone-Brewer wrote:
>Whoops - thanks for this! I had the facts back to front.
>Well, this is good news in that the KJV tagging is better than the 
>NASB, but bad news in that the NASB is closer to the ESV than the KJV is.
>
>Thanks for warning us before we did too much work.
>
>David IB
>
>At 23:34 19/03/2011, Troy A. Griffitts wrote:
>>Nice job guys.  Just a point of clarification:
>>
>>On 03/19/2011 01:04 PM, David Instone-Brewer wrote:
>>
>> > 4) merge the resultant text with the verb parsing in the tagged KJV
>>
>>I'm confused a bit about where the NASB and KJV come into play with 
>>your tagging efforts.
>>
>>
>> > Since starting this, I've heard from Troy who originally 
>> organised the team who tagged the NASB.
>> > He says his method is:
>>
>>We did not tag the NASB.  We tagged the KJV.  I would not use the 
>>NASB markup if I was doing this project, to avoid any copyright 
>>infringement of Lockman's data.
>>
>>
>>Troy
>>
>>
>>
>>
>>On 03/19/2011 09:54 PM, David Instone-Brewer wrote:
>>>Dear Rob
>>>
>>>I've been doing some experiments with Gen.1 to work out a system.
>>>I've found a method which works really well - the whole tagging of 
>>>Gen.1 has been done correctly by automatic comparisons and it has 
>>>only gone wrong in a few verses.
>>>I've tried using Stanfords parsing engine at 
>>><http://nlp.stanford.edu:8080/parser/>http://nlp.stanford.edu:8080/parser/
>>>but this didn't fix it. I've attached a file listing my 
>>>experiments and their results.
>>>
>>>I think what would fix it is a semantic domain dictionary. What's 
>>>happened is that the two versions are too different in v. 11:
>>>
>>>ESV: And God said, "Let the earth sprout vegetation, plants 
>>>yielding seed, and fruit trees bearing fruit in which is their 
>>>seed, each according to its kind, on the earth." And it was so.
>>>NASB: Then God said, "Let the earth sprout vegetation: plants 
>>>yielding seed, and fruit trees on the earth bearing fruit after 
>>>their kind with seed in them"; and it was so.
>>>
>>>The change in order in the words in bold makes it too difficult 
>>>for the comparison program to match things up.
>>>
>>>I think we will need humans at these points, but I think we can 
>>>highlight the likely places where problems exist.
>>>
>>>Tomorrow I'll have a go at producing the whole text of Genesis, so 
>>>you have some data to play with
>>>
>>>David IB
>>>
>>>
>>>============
>>>THe process we are attempting is:
>>>
>>>1) convert the NASB XML text to something which looks like a 
>>>BibleWorks exported text
>>>   (ie each verse on one line starting with a simple ref (eg Gen 
>>> 1:1 In the beginning...)
>>>
>>>2) use the Word 2003+ text comparison tools (which are much 
>>>superior to Word 97) to compare the text of both versions 
>>>producing something like:
>>>Gen 1:2  <w H776>The earth</w> was formless <w H8414>formless</w> 
>>><w H922>and void</w>, and <w H2822>darkness </w> <w H5921>was 
>>>over</w> the <w H6440>surface </w> <w H8415>of the deep</w>, and . 
>>>And <w H7307>the Spirit </w> <w H430>of God</w> was <w 
>>>H7363!b>moving hovering </w> <w H5921>over</w> the <w 
>>>H6440>surface </w> <w H4325>of the waters.  </w>.
>>>
>>>3) create a site where human can easily correct this automatic markup
>>>  - eg the proof of concept 
>>> <http://www.slowley.com/tagger-proof-of-concept/example.html>here.
>>>
>>>4) merge the resultant text with the verb parsing in the tagged KJV
>>>
>>>Since starting this, I've heard from Troy who originally organised 
>>>the team who tagged the NASB. He says his method is:
>>>
>>>1) starts with a lemma tagged text, the KJV, and CrossWay's ESV 
>>>data in OSIS format.
>>>2) the ESV module is iterated each verse at a time and is 
>>>processed as such:
>>>3) the OSIS markup is stripped from the ESV text and positioning 
>>>information is retained
>>>4) a word table is built from the KJV text:
>>>        KJV Word 1    |    Strongs #
>>>        KJV Word 2    |    Strongs #
>>>5) a second table is build from the ESV text:
>>>        ESV Word 1    |
>>>        ESV Word 2    |
>>>6) these tables are passed to a function which is responsible 
>>>solely for the logic to fill in the second part of the second 
>>>table with Strong's numbers.
>>>7) the returned table is used to reconstitute the the OSIS tags to 
>>>the ESV text including word-level Strong's markup.
>>>See a screenshot for the community collaboration tool for KJV 
>>>Strongs markup project is at 
>>><http://crosswire.org/sword/kjv2003/#ss>http://crosswire.org/sword/kjv2003/#ss
>>>We're hoping to convert it to a web application instead of a 
>>>standalone Java GUI, but that hasn't happened yet.
>>>I'd love to work together on this effort.  Please keep me posted 
>>>on any progress and let me know if I can help in anyway.
>>>Troy
>>>
>>>
>>>
>>>
>>>At 10:18 17/03/2011, Robert Slowley wrote:
>>>>So, presumably if you could script it to break each chapter in to a
>>>>separate file, do the comparisons, and then re-export as a single file
>>>>we could import that in to a tool like mine so a human could fix the
>>>>errors and do the bits the auto-comparison failed to do.
>>>>
>>>>On Tue, Mar 15, 2011 at 8:19 AM, David Instone-Brewer
>>>><mailto:davidinstonebrewer at gmail.com><davidinstonebrewer at gmail.com> wrote:
>>>> > From the automatic comparisons produced by Word, we get:
>>>> >
>>>> > Gen 1:1  <w H7225>In the beginning,</w> <w H430>God</w> <w
>>>> > H1254!a>created</w> <w H8064>the heavens</w> <w H776>and the earth </w>.
>>>> > Gen 1:2  <w H776>The earth</w> was <w H8414>without form</w> <w H922>and
>>>> > void</w>, and <w H2822>darkness</w> <w H5921>was over</w> the <w
>>>> > H6440>face</w> <w H8415>of the deep</w>. And <w H7307>the Spirit</w> <w
>>>> > H430>of God</w> was <w H7363!b>hovering </w> <w H5921>over</w> the <w
>>>> > H6440>face</w> <w H4325>of the waters  </w>.
>>>> >
>>>> > - ie the first two verses are already perfectly tagged. In 
>>>> fact there aren't
>>>> > any problems in Gen.1 till we get to v.5:
>>>> >
>>>> > Gen 1:5  <w H430>God</w> <w H7121>called</w> <w H216>the light</w> <w
>>>> > H3117>Day</w>, <w H2822>and the darkness</w> <w H7121>he called</w> <w
>>>> > H3915>Night.</w>. And <w H6153>there was evening</w> <w 
>>>> H1242>and there was
>>>> > morningthe first</w>, <w H259>one</w> <w H3117>day</w>.
>>>> >
>>>> > The problem is that Word gives up making these comparisons after a few
>>>> > chapters.
>>>> > Some of these problems can be cleared up by macros.
>>>> >
>>>> > David IB
>>>> >
>>>> > At 00:43 15/03/2011, Robert Slowley wrote:
>>>> >
>>>> >> I think I can produce a better text to produce something 
>>>> which has less to
>>>> >> correct.
>>>> > What do you mean?
>>>> >
>>>> >> It would be useful to have transliterated Hebrew and a 
>>>> single-word meaning
>>>> >> instead of the numbers.
>>>> > I have an electronic copy of the stuff you get on popups on
>>>> > 
>>>> http://classic.net.bible.org/verse.php?search=Genesis%201:30&book=genesis&chapter=1&verse=30 
>>>>
>>>> > for Strongs already - which I was planning to integrate. If the
>>>> > numbers are replaced with 'transliterated Hebrew' or a 'single-word
>>>> > meaning' what specifically would that mean?
>>>> >
>>>> > For instance on
>>>> > 
>>>> http://classic.net.bible.org/verse.php?search=Genesis%201:30&book=genesis&chapter=1&verse=30 
>>>>
>>>> > for the strongs reference h03651, which is the transliterated hebrew,
>>>> > and which is the single word meaning?
>>>> >
>>>> >> It would be useful to divide the top line by the tagging, not by any
>>>> >> English
>>>> >> parsing
>>>> >>  eg Gen.1.30  || and to every thing (h3605 )||
>>>> >>   instead of     || and to every (h3605) ||  thing (h3605 ) ||
>>>> > In the case of Genesis 1:30 the text behind it is:
>>>> > NASB: ... <w H3605>and to every</w> <w H3605>thing</w> ...
>>>> >
>>>> > Presumably there is a reason for the text to have two separate sets of
>>>> > words both tagged individually with H3605? Or is it just a markup
>>>> > error?
>>>> >
>>>> > Presumably in some cases it words should be merged if they have the
>>>> > same strongs and are next to each other, but in other cases, this
>>>> > isn't the case, e.g. Isa 6:3
>>>> > 
>>>> http://classic.net.bible.org/verse.php?search=isa%206:3&book=isa&chapter=6&verse=3 
>>>>
>>>> >
>>>> > Has:
>>>> >
>>>> > <w H6918>Holy</w>, <w H6918>Holy</w>, <w H6918>Holy</w>, is the <w
>>>> > H3068>Lord</w> <w H6635>of hosts</w>
>>>> >
>>>> > because the Hebrew has swdq repeated 3 times, and I assume that the
>>>> > reader who understands Strong's gets this indication by it being
>>>> > repeated rather than there being <w H6918>Holy, Holy, Holy</w>. Is
>>>> > that right?
>>>> >
>>>> >> It might be better to have the bottom line with a separate box for very
>>>> >> word. Sometimes we will want to divide things up differently
>>>> > As I see it we have 'phrases' (a set of one or more words) which may
>>>> > have one or more strongs references. In some cases a set of words with
>>>> > have a shared strongs reference, but in other cases like Isa 6:3 sets
>>>> > of contiguous words may have the same strongs references but still be
>>>> > separate 'phrases'. As I see it there's no automatically working this
>>>> > out.
>>>> >
>>>> > What I was thinking was to have some algorithm that tries to
>>>> > automatically map the NASB strongs annotations on to the ESV text,
>>>> > similar to what I have already crudely done here. That can either try
>>>> > to group things as the NASB does (where a set of contiguous words
>>>> > share a strongs reference), or do what I have done here (which is
>>>> > easier) which is to automatically group words in to a 'phrase' where
>>>> > they share the same strongs references.
>>>> >
>>>> > Either way not all of the ESV can be automatically annotated in this
>>>> > way, the annotation will be wrong in some cases, and the automated
>>>> > grouping may be wrong in some cases. So I was thinking of making the
>>>> > interface such that once the automated grouping has been attempted the
>>>> > end user can click on a box which will make it selected, then click on
>>>> > the next box to the left or right (and so on), when this is done a
>>>> > button for "merging in to a phrase" would appear - then if this is
>>>> > clicked they would be made in to a phrase and could have their strongs
>>>> > references assigned. Alternatively clicking on a box that represents a
>>>> > phrase of one or more words will cause a "demerge" button to appear
>>>> > that will separate out all the words. This will allow the end user to
>>>> > handle both types of situation.
>>>> >
>>>> > I also thought some sort of "This verse is tagged correctly" button
>>>> > would be good. In some cases the program will annotate everything, but
>>>> > it will still need to be checked by a human - and a human may wish
>>>> > their annotation to be checked by someone else for quality purposes.
>>>> > When a verse is marked as correct, it can have a tick or something,
>>>> > and there can be a page of "verses that need work" which it would
>>>> > automatically be removed from. Does that sound sensible?
>>>> >
>>>> > We have easy access to the SBLGNT (with apparatus) and Leningrad
>>>> > Codex. Is it worthwhile including those for each verse? I don't know
>>>> > what process an annotator would go through, and what level of
>>>> > knowledge of the original languages they would use.
>>>> >
>>>> > I worked a bit today on tidying up the classes I've written, and
>>>> > improving the processing of the text (in the next few weeks I'll send
>>>> > you a list of the suspicious stuff I found while processing your files
>>>> > ;-) ). I'm away next week for my 1st year's anniversary holiday - but
>>>> > after that can start work on making this in to an actual web app that
>>>> > would be useful rather than a static web page demo of the sort of
>>>> > thing I had in mind.
>>>> >
>>>> > Any thoughts / comments / ideas appreciated!
>>>> >
>>>> > It'd probably be a good idea to see if we can improve the automatic
>>>> > annotation of the ESV from the NASB if we can, as any progress made
>>>> > here before people start manually annotating / checking will reduce
>>>> > the amount of man hours needed to complete the task.
>>>> >
>>>> > -Rob
>>>> > --
>>>> > http://www.slowley.com/
>>>> >
>>>> > "On two occasions, I have been asked [by members of Parliament],
>>>> > 'Pray, Mr. Babbage, if you put into the machine wrong figures, will
>>>> > the right answers come out?' I am not able to rightly apprehend the
>>>> > kind of confusion of ideas that could provoke such a question."
>>>> > -- Charles Babbage (1791-1871)
>>>>
>>>>
>>>>
>>>>--
>>>>http://www.slowley.com/
>>>>
>>>>"On two occasions, I have been asked [by members of Parliament],
>>>>'Pray, Mr. Babbage, if you put into the machine wrong figures, will
>>>>the right answers come out?' I am not able to rightly apprehend the
>>>>kind of confusion of ideas that could provoke such a question."
>>>>-- Charles Babbage (1791-1871)
>>>
>>>
>>>
>>>
>>>
>>>_______________________________________________
>>>tyndale-devel mailing list
>>><mailto:tyndale-devel at crosswire.org>tyndale-devel at crosswire.org
>>>http://www.crosswire.org/mailman/listinfo/tyndale-devel
>>
>>_______________________________________________
>>tyndale-devel mailing list
>>tyndale-devel at crosswire.org
>>http://www.crosswire.org/mailman/listinfo/tyndale-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.crosswire.org/pipermail/tyndale-devel/attachments/20110320/b02e7bb1/attachment-0001.html>

More information about the tyndale-devel mailing list