[sword-devel] Titles/preverse mess rant

[Troy A. Griffitts]

Dear all,

Beginning with around r2688 we added an OSIS reference document and test 
to our testsuite.

For those not familiar with the testsuite, after building source,

cd tests
make
cd testsuite
./runall.sh

This will run the engine through the entire testsuite.  To run just a 
single test, e.g.,

./runtest.sh osis

Adding to the testsuite is stupid easy.  Anything you'd like to do is 
added to one of the existing tests scripts, and the expected output is 
added to the .good file.  Or to add a completely new test, you would 
create 2 new files, for example:

mytest.sh
__________
#!/bin/sh
echo This is the expected output


mytest.good
___________
This is the expected output


That's it.

Have a look at the osis script (osis.sh) and expected output 
(osis.good).  I believe it does an osis2mod on a reference OSIS document 
provided by Daniel Owens with input from Patrick Zimmermann, then calls 
a C++ program in the tests/ folder to run some SWORD API methods against 
that text.

If we could check the OSIS reference document for a representative 
preverse section (or add one if it doesn't exist) and then check the 
expected output, it would go a long way to staging a definitive problem 
case and desired result for one of us to solve.

Thanks for the emails on this one and hope we can funnel the energy into 
a solution.

Troy




On 06/22/2012 10:38 AM, Peter von Kaehne wrote:
> Hi guys,
>
> I know that it never comes in good to moan publicly.
>
> But the latest thread, started by Karl is yet again another aspect of
> the total mess the preverse setup is.
>
> And it covers a multitude of angles and probably several programmes.
>
> And it has been highlighted again and again by various module makers,
> reaching back several years, with very little progress, in fact, I would
> think with very obvious new regressions.
>
> As a module maker this is hugely embarrassing for me. And I have no way
> of fixing it as I am not even clear what all is involved.
>
> What I can say is that there is essentially very little reliable and
> predictable output coming out of any number of ways to encode
> pre/interverse content.
>
> So as a result, we can see titles moving a verse higher than they should
> be - either (and that is bad) being appended to end of the last verse or
> (and that is unbelievably worse) being prepended to the last verse!!
>
> Further, we can see titles appearing doubled up (see Karl's recent
> thread) and we can see titles appearing/vanishing in unpredictable
> fashion to the title filter.
>
> What I do know is that
>
> a) modules compiled with older versions of osis2mod generally fare better.
> b) some of the output of usfm2osis.pl is not helpful (interleaving of
> verse sID (new verse) and verse eID (last verse) tags)
> c) even old modules suffer from title duplication (see Karl's thread)
> d) the filter system is not completely error free
> e) not all frontends deal with preverse filter output in the same sane way
>
> Now a lot of this is known, well known, regurgitated and remasticated
> for several years now. I am quite sure it would require not a huge
> amount of actual individual programming effort to get right. But it
> would require a joint effort and a joint willingness to see this as a
> shared problem instead of the repeated passing the buck stuff I got fed
> up seeing here.
>
> As I say, as a reasonably prolific module maker with some limited
> programming skills it is way out of my ability to solve by myself.
> Believe me, I have tried. I have learned some XML skills and I have
> added and subtracted attributes, shunted tags about and what not else.
> But that is not really all of it as there is essentially no clearcut and
> definable input which will produce a correct output.
>
> We do need to solve it. We are messing the people who rely on us (Bible
> publishers, specifically) endlessly about and we can do so only for so long.
>
> Right now I am holding back on 4 texts which I can not fix properly
> (Turkish, Azeri, Fijian and Pohnpeian). The former two are for people
> groups counting to several dozens of millions and having electronic
> bibles available makes a huge difference.
>
> So, please, take this seriously.
>
> Peter
>
> _______________________________________________
> sword-devel mailing list: sword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page