[sword-devel] osis2mod linking bug

Greg Hellings greg.hellings at gmail.com
Thu Sep 4 16:44:19 MST 2008 

On Thu, Sep 4, 2008 at 6:40 PM, Peter von Kaehne <refdoc at gmx.net> wrote:
> I am not sure whteher this should be raised here, but I think it is connected with the problem.
>
> A fair number of translations reorder occasionally verses. usually praraphrasic translations, but occasionally even fairly accurate ones.
>
> Currently osis2mod reorders these verses back again, which is probably fine considering how few verses these actually are, but I think it should be eventually addressed too.

I imagine that this is either already addressed by or will at least
require the updates of the VerseTreeKey, but you're right - it should
be addressed.

--Greg

>
> Peter
>
>
> -------- Original-Nachricht --------
>> Datum: Fri, 5 Sep 2008 09:26:14 +1000
>> Von: "Ben Morgan" <benpmorgan at gmail.com>
>> An: "SWORD Developers\' Collaboration Forum" <sword-devel at crosswire.org>
>> Betreff: Re: [sword-devel] osis2mod linking bug
>
>> As to checking for (consecutive) linked verses, you can do something like
>> this (horrible, but should work in theory - I haven't tested it):
>>
>> // keep the key we were at
>> SWKey* k = module.getKey().clone();
>>
>> // move forward without skipping over links
>> module.setSkipConsecutiveLinks(false);
>> module.increment();
>> SWKey* k_2 = module.getKey().clone();
>>
>> // now go back and try again skipping over links
>> module.setKey(k)
>> module.setSkipConsecutiveLinks(true);
>> module.increment();
>>
>> // If k != module.getKey(), we skipped over some links
>> boolean linked = !(module.getKey().equals(k2));
>>
>> I don't believe you can easily check for non-consecutive verses being
>> linked
>> - not that it makes all that much sense.
>> God Bless,
>> Ben
>> -------------------------------------------------------------------------------------------
>> The Lord is not slow to fulfill his promise as some count slowness,
>> but is patient toward you, not wishing that any should perish,
>> but that all should reach repentance.
>> 2 Peter 3:9 (ESV)
>>
>>
>> On Fri, Sep 5, 2008 at 8:14 AM, Troy A. Griffitts
>> <scribe at crosswire.org>wrote:
>>
>> > Right, it seems like we might need a bool SWModule::isLinked(const SWKey
>> > &) which tells you if the current location is linked to the provided
>> > location.  Something which compares 'inodes' in versetext and @link
>> > values in others, etc...
>> >
>> > That's doable from the engine point of view, but I'm not sure will solve
>> > your problem.
>> >
>> > Like you mention, you're looking for a way to get the entire set of
>> > entries that link to a data slot.  Something like:
>> >
>> > find / -samefile xyz -print
>> >
>> > I guess you could use the newly proposed method above to iterate all
>> > entries and check if any are the 'samefile' as one you are interested
>> > in.  But like the filesystem, we don't store a set of all directory
>> > entries that happen to point to the same inode.  And if you're looking
>> > for backward compatability, we can't change the format (not that I would
>> > want to change the format for this anyway).
>> >
>> > With the the logic osis2mod is using (for any verses outside of KJV,
>> > read text from closest previous valid KJV entry, append new verse text
>> > and rewrite the entry) it sounds these exception KJV-reversifications
>> > should probably do the ugly: `find / -samefile xyz -print` logic to
>> > check if the entry they are about to rewrite is linked to by anyone
>> > else.  It sucks, but it is all I can think of that would solve this
>> > problem.  And pragmatically, you probably only have to: do { module--; }
>> > while (!module.Error() && module.getRawEntryBuf() ==
>> > aboutToBeRewrittenEntryBuf);
>> >
>> > Hope I haven't pass the buck for no good reason.
>> >
>> >        -Troy.
>> >
>> > DM Smith wrote:
>> > > Greg Hellings wrote:
>> > >> DM,
>> > >>
>> > >> On Thu, Sep 4, 2008 at 1:05 PM, DM Smith <dmsmith555 at yahoo.com>
>> wrote:
>> > >>
>> > >>> I'm trying to solve an osis2mod linking bug that was exposed by
>> several
>> > >>> beta modules.
>> > >>>
>> > >>> Here is the scenario:
>> > >>> <verse osisID="XXX.1.29 XXX.1.30 XXX.1.31">Text for the last three
>> > >>> verses of XXX, chapter 1 in the KJV versification</verse>
>> > >>> <verse osisID="XXX.1.32 XXX.1.33">Text for additional verses in the
>> > XXX,
>> > >>> chapter 1 in the non-KJV versification</verse>
>> > >>>
>> > >>> 1) osis2mod links 1.30 and 1.31 by writing out the start and offset
>> of
>> > >>> the data for 1.29. That means that 1.29 has to be written first. In
>> the
>> > >>> index file the result is that all three entries have the same start
>> and
>> > >>> offset. So far so good, if there weren't a bug that writes the links
>> > >>> first and then the data, resulting in start/offset of 0/0 for the
>> > links.
>> > >>> But I've already figured out how to fix that bug.
>> > >>>
>> > >>> 2) Now a non-KJV verse is found and osis2mod appends it to the last
>> > >>> verse of the chapter according to the KJV versification. So the
>> entry
>> > >>> for XXX.1.31 is found, the raw text is gotten and it is appended and
>> > >>> this augmented text is written to the data file, at the end of the
>> > file.
>> > >>> Finally the index entry for XXX.1.31 is updated.
>> > >>>
>> > >>> The problem is that 1.29 and 1.30 are not updated, they still point
>> to
>> > >>> the "1.29-1.31" text and now 1.31 points to the "1.29-1.33" text.
>> > >>>
>> > >>> 3) The other problem is with 1.33, it is noticed that it is out of
>> > >>> bounds and is changed to 1.31 and then it is linked to 1.32, which
>> does
>> > >>> not exist and thus 1.31 is re-written to 0/0.
>> > >>>
>> > >>> I'm looking for a way to solve these problems. Here is what I am
>> > >>> thinking and I'd like feedback or a better way.
>> > >>> For 1) I have postponed the writing of the links until the verse is
>> > >>> written. I could either wait until the next verse is ready to be
>> > >>> written, or later. I've decided to wait until the very end and do
>> the
>> > >>> linking then. This might help solve 2 and 3.
>> > >>>
>> > >>> For 2) I'm thinking that the verse to append is the last verse in
>> the
>> > >>> chapter with content. By postponing linking until later, it will
>> append
>> > >>> to 1.29. Then the linking will propagate the final start/offset
>> values
>> > >>> for 1.29. The problem I have with this is that this is somewhat disk
>> > >>> intensive. I start with the last verse in the chapter and get the
>> raw
>> > >>> text for it. If there is none, I then decrement and refetch until I
>> > find
>> > >>> it. I looked for a way to know if a verse were part of a linked set
>> and
>> > >>> what the members of that set were, but I didn't see any in the SWORD
>> > >>> engine. Am I missing it?
>> > >>>
>> > >> When I ran into this bug in dabbling with a SWORD front-end, I was
>> > >> told that the only way to test for whether it is part of a set is to
>> > >> compare the text of verse x with verse x-1.  I don't know that the
>> > >> feature we both sought has been added.  If it has, I haven not heard
>> > >> word of it.
>> > >>
>> > > Ouch, that is expensive. It should not be necessary to dig down to the
>> > > text and then do a string comparison. Comparing the start/offset
>> > > (block/start/offset for compressed) is sufficient. I don't see how to
>> > > get this info. It would be nice to have the ability, in the engine, to
>> > > compare two keys to see if they refer to the same text.
>> > >
>> > >> Alternatively -- aren't we supposed to be moving to the VerseTreeKey?
>> > >> Shouldn't a new module in the Beta stage be using that?  It seems
>> like
>> > >> that would completely do away with the problem.  Updating osis2mod to
>> > >> use that, either as the default or as an option for a module like
>> this
>> > >> might eliminate the issue?
>> > >>
>> > > I think ultimately that osis2mod will need to be updated to handle
>> other
>> > > versifications. The VerseKey module type is significantly faster than
>> a
>> > > VerseTreeKey module will ever be.
>> > >
>> > > My goal in modifying osis2mod is that it will create 1.5.9 compatible
>> > > modules.
>> > >
>> > > Later, I intend to make modifications that will require 1.5.12.
>> > > (Specifically, an extension of preverse content that we've discussed
>> > > here that will have a preverse div and not just a preverse title.) At
>> > > that time, it would also be appropriate to handle VerseTreeKey as an
>> > option.
>> > >>
>> > >>> For 3) this is fairly simple, one should not link verses that are
>> not
>> > in
>> > >>> the KJV versification.
>> > >>>
>> > >> That seems like a very stringent restriction.  If linking is
>> supported
>> > >> for the KJV system, why should it not be allowed for other systems?
>> > >> It seems like linking should be fixed in a way to allow everyone to
>> do
>> > >> it.
>> > > It's not a stringent restriction for the current osis2mod which only
>> > > works for KJV versifications. Later, when osis2mod is upgraded then
>> that
>> > > restriction will need to be removed.
>> > >
>> > > In Him,
>> > >     DM
>> > >
>> > >
>> > >
>> > > _______________________________________________
>> > > sword-devel mailing list: sword-devel at crosswire.org
>> > > http://www.crosswire.org/mailman/listinfo/sword-devel
>> > > Instructions to unsubscribe/change your settings at above page
>> >
>> >
>> > _______________________________________________
>> > sword-devel mailing list: sword-devel at crosswire.org
>> > http://www.crosswire.org/mailman/listinfo/sword-devel
>> > Instructions to unsubscribe/change your settings at above page
>> >
>
> --
> GMX startet ShortView.de. Hier findest Du Leute mit Deinen Interessen!
> Jetzt dabei sein: http://www.shortview.de/wasistshortview.php?mc=sv_ext_mf@gmx
>
> _______________________________________________
> sword-devel mailing list: sword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page
>

More information about the sword-devel mailing list