[sword-devel] footnote analysis problem (Re: markup within ThML <div></div> not interpreted)

[Karl Kleinpaste]

DM Smith <dmsmith555 at yahoo.com> writes:
> The problem is also in the OSIS filters. Both for the web and for  
> BibleCS. I notice that <divineName>Lord</divineName> is not being  
> rendered properly, like it is elsewhere. I think this is a bug.

As a couple folks know, I found/fixed the first part of the essential
problem with this, discussed in freenode #sword and with a copy sent
to patches@ last evening.  The problem was that all tokens in title
and div headers were being dropped by {osis,thml}headings.cpp.

Fixing that was the easy part.  Last evening, I also fixed how
GnomeSword handles the display of preverse headers, with scribe's
advice on getting preverse content passed through RenderText().  But
unfortunately what has been fixed is merely display, while content of
what the display references is broken.  Below is some email I wrote
late last evening to gnomesword-developers which someone suggested
should more appropriately be sent here -- this is a deeper problem for
Sword as a whole, not just particular frontends.
________________________________________________________________

Consider this verse that includes <div>, and lookup's analysis of it:

<div class="sechead">The Creation<br />test text <note>here is footnote text</note> more <i>test</i> text<scripRef>Ps 1:1</scripRef></div><scripRef>Ps 102:25; Is 40:21; John 1:1, 2; Heb 1:10</scripRef>In the beginning <scripRef>Ps 89:11; 90:2; Acts 17:24; Rom 1:20; Heb 11:3</scripRef>God <scripRef>Job 38:4; Is 42:5; 45:18; Rev 4:11</scripRef>created the heavens and the earth.

[ Footnote ]
        [ 1 ]
                body = Ps 102:25; Is 40:21; John 1:1, 2; Heb 1:10
                refList = Psalms 102:25; Isaiah 40:21; John 1:1; John 1:2; Hebrews 1:10
        [ 2 ]
                body = Ps 89:11; 90:2; Acts 17:24; Rom 1:20; Heb 11:3
                refList = Psalms 89:11; Psalms 90:2; Acts 17:24; Romans 1:20; Hebrews 11:3
        [ 3 ]
                body = Job 38:4; Is 42:5; 45:18; Rev 4:11
                refList = Job 38:4; Isaiah 42:5; Isaiah 45:18; Revelation of John 4:11
        [ count ]
                value = 3
[ Heading ]
        [ 0 ]
                class = sechead
        [ Preverse ]
                0 = The Creation<br />test text <note>here is footnote text</note> more <i>test</i> text<scripRef>Ps 1:1</scripRef>

Notice that Sword has found 3 footnotes, all of which are found in the
verse text.  There are 2 more notes that are part of the header, but
they aren't indexed by Sword in the usual way.  So far, they're just
text that has been copied to the extracted header content.

Along comes RenderText() on the extracted preverse header.  It generates:

The Creation<br />test text <a href="passagestudy.jsp?action=showNote&type=n&value=1&module=NASB&passage=Genesis+1%3A1"><small><sup>*n</sup></small></a>  more <i>test</i> text<a href="passagestudy.jsp?action=showNote&type=x&value=2&module=NASB&passage=Genesis+1%3A1"><small><sup>*x</sup></small></a> 

Notice that in the converted href's, there is mention of "value=", for
1 and 2.  But those are indices into the wrong thing: They refer to
the set of 3 footnotes found in the verse text.  When this is
displayed, it correctly shows up as *n and *x, but the hover text that
actually appears in GS' preview area is not the content that was
intended from <div>; rather, it's the text of the first two footnotes
from verse text.  The *n displays "Ps 102:25; Is 40:21; John 1:1, 2;
Heb 1:10" as plain text (not hotlinked) in preview, and the *x is a
repeat hotlink of the 2nd scripRef.

The original "here is footnote text" and "Ps 1:1" content from the
<div> are just lost.  They might as well have never been there.

One additional odd result is that, just for the self-referential
weirdness factor, for verses in which the <div> header contains a
<note> but there is no <note> in the verse text, the text shown in
preview on hover is the actual verse text.

I haven't the first idea how to resolve this conflict.  Sword wants to
index notes+xrefs out of verse text _only_.  But it seems completely
reasonable to me that the header should be able to contain notes+xrefs
as well, because in fact some texts legitimately have them.

Comment?
________________________________________________________________

I believe that the real solution, which DM mentioned very late last
evening in #sword, is to require that footnote analysis be carried out
so as to include items in OSIS title and ThML div sections, as well as
elsewhere, so that the indices generated by RenderText() are correct
everywhere.