[sword-devel] osis2mod bug - PLEASE HELP

[Brian J Dumont]

Hi all,

I've struggled with large commentaries before.  It always seemed that 
once commentary sections get too big, then random sections of text start 
disappearing.

I'm currently almost done repackaging a part of the EarlyFathers module 
to be the homilies of John Chrysostom in commentary form.  Thing is, his 
homilies are long and I'm hitting the same problems again.  Previously, 
I've been able to split sections between verses to awkwardly get things 
to work.  This time I can't; some of these large homilies are on a 
single verse, and have another homily on the next.

So this time I've had no choice but to give up or to buckle down and get 
something that the developers can really bite into.  I've created an 
example OSIS module.  One section: Matt.1.1.  No special characters, no 
footnotes, no headers, almost no markup at all really.

I'm not compiling it into a compressed format.  My command/output is:
[bjdasc at ascpc5] osis2mod mod debug.osis.xml
You are running osis2mod: $Rev: 2671 $
SUCCESS: osis2mod: has finished its work and will now rest

I can't always predict *exactly* at what size it will fail (it seems to 
depend on things like number of paragraph markers, etc).  But it always 
fails when the section gets to be roughly 64,000 - 65,000 characters.  
This is a pretty suspicious number, being just below 2^16.  I assume 
that I can't exactly predict the size because there's some overhead, 
some amount for paragraph markers, header, etc.

The attached example file fails (I've also included the conf file that 
I'm using to make life easier for any developer that wishes to look at 
this).  The output is drastically shorter than the input.  Please delete 
one character from the text in Matt.1.1.  It all works nicely.

This is not a front end problem.  I can reproduce the problem in either 
Xiphos or diatheke.  My diatheke command has been:
diatheke -b Chrysostom  -k "Mt 1:1"

But you can even see when it has failed by the size of the "nt" file 
created by osis2mod.  This osis file, as attached, creates an nt file 
that is 70 bytes.  Delete one character and it goes up to 65607 bytes.  
Note that 70 bytes + 2^16 bytes = 65606 bytes.

PLEASE PLEASE PLEASE PLEASE ... I beg you ... I'm stuck without a fix to 
this ... it seems like it's an array size limit or a compiler size limit 
for some data type or something like that.  If you need some legwork, 
then let me know; I'll do it.

Thanks in advance,
Brian


-- 
-----------------------------------------------------------------------
The question is not "Have you accepted Jesus into your heart", but "Has
Jesus accepted you into His heart?"; not "have you given your whole
life for God", but "has God given His whole life for you"
	- Pr Bryan Wolfmueller, 2010

-------------- next part --------------
A non-text attachment was scrubbed...
Name: debug.osis.xml
Type: text/xml
Size: 65739 bytes
Desc: not available
URL: <http://www.crosswire.org/pipermail/sword-devel/attachments/20120302/b485d920/attachment-0001.xml>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: chrysostom.conf
URL: <http://www.crosswire.org/pipermail/sword-devel/attachments/20120302/b485d920/attachment-0001.ksh>