[sword-devel] osis2mod bug - PLEASE HELP

[Brian J Dumont]

That would be super-keen DM.  It would be nice if it also noted the need 
for ModDrv "RawCom4" to go along with "-s 4"

Thanks,
Brian

On 03/02/2012 08:15 AM, DM Smith wrote:
> I think osis2mod should detect the problem, output an error and a clear suggestion to use -s 4.
>
> Cent from my fone so theer mite be tipos. ;)
>
> On Mar 2, 2012, at 8:06 AM, Greg Hellings<greg.hellings at gmail.com>  wrote:
>
>> On Fri, Mar 2, 2012 at 7:02 AM, Brian J Dumont<brian.j.dumont at gmail.com>  wrote:
>>> Hi all,
>>>
>>> I've struggled with large commentaries before.  It always seemed that once
>>> commentary sections get too big, then random sections of text start
>>> disappearing.
>>>
>>> I'm currently almost done repackaging a part of the EarlyFathers module to
>>> be the homilies of John Chrysostom in commentary form.  Thing is, his
>>> homilies are long and I'm hitting the same problems again.  Previously, I've
>>> been able to split sections between verses to awkwardly get things to work.
>>>   This time I can't; some of these large homilies are on a single verse, and
>>> have another homily on the next.
>>>
>>> So this time I've had no choice but to give up or to buckle down and get
>>> something that the developers can really bite into.  I've created an example
>>> OSIS module.  One section: Matt.1.1.  No special characters, no footnotes,
>>> no headers, almost no markup at all really.
>>>
>>> I'm not compiling it into a compressed format.  My command/output is:
>>> [bjdasc at ascpc5] osis2mod mod debug.osis.xml
>>> You are running osis2mod: $Rev: 2671 $
>>> SUCCESS: osis2mod: has finished its work and will now rest
>>>
>>> I can't always predict *exactly* at what size it will fail (it seems to
>>> depend on things like number of paragraph markers, etc).  But it always
>>> fails when the section gets to be roughly 64,000 - 65,000 characters.  This
>>> is a pretty suspicious number, being just below 2^16.  I assume that I can't
>>> exactly predict the size because there's some overhead, some amount for
>>> paragraph markers, header, etc.
>>>
>>> The attached example file fails (I've also included the conf file that I'm
>>> using to make life easier for any developer that wishes to look at this).
>>>   The output is drastically shorter than the input.  Please delete one
>>> character from the text in Matt.1.1.  It all works nicely.
>>>
>>> This is not a front end problem.  I can reproduce the problem in either
>>> Xiphos or diatheke.  My diatheke command has been:
>>> diatheke -b Chrysostom  -k "Mt 1:1"
>>>
>>> But you can even see when it has failed by the size of the "nt" file created
>>> by osis2mod.  This osis file, as attached, creates an nt file that is 70
>>> bytes.  Delete one character and it goes up to 65607 bytes.  Note that 70
>>> bytes + 2^16 bytes = 65606 bytes.
>>>
>>> PLEASE PLEASE PLEASE PLEASE ... I beg you ... I'm stuck without a fix to
>>> this ... it seems like it's an array size limit or a compiler size limit for
>>> some data type or something like that.  If you need some legwork, then let
>>> me know; I'll do it.
>>>
>>> Thanks in advance,
>>> Brian
>>>
>>>
>>> --
>>> -----------------------------------------------------------------------
>>> The question is not "Have you accepted Jesus into your heart", but "Has
>>> Jesus accepted you into His heart?"; not "have you given your whole
>>> life for God", but "has God given His whole life for you"
>>>         - Pr Bryan Wolfmueller, 2010
>>>
>>>
>>> _______________________________________________
>>> sword-devel mailing list: sword-devel at crosswire.org
>>> http://www.crosswire.org/mailman/listinfo/sword-devel
>>> Instructions to unsubscribe/change your settings at above page
>> Brian,
>>
>> Try adding the switch '-s 4' to your import. Also change your driver
>> in the conf file to RawCom4. This will increase the per-entry size
>> limit from 2^16 bytes (the default) to 2^32 bytes giving you an
>> effective limit of 2GB for each verse or other entry.
>>
>> --Greg
>>
>> _______________________________________________
>> sword-devel mailing list: sword-devel at crosswire.org
>> http://www.crosswire.org/mailman/listinfo/sword-devel
>> Instructions to unsubscribe/change your settings at above page


-- 
-----------------------------------------------------------------------
The question is not "Have you accepted Jesus into your heart", but "Has
Jesus accepted you into His heart?"; not "have you given your whole
life for God", but "has God given His whole life for you"
	- Pr Bryan Wolfmueller, 2010