[sword-devel] Segfault in LZSS code

[Troy A. Griffitts]

Maybe I don't understand. It is the default because it is the first compression type we supported and was the compression type of STEP data files (which previously was an open standard for Bible data; not having anything to do with the STEP Bible Project).

The problem would be the same if ZIP was the default and you gave the ZIP compression driver LZSS data files.

In summary, if you specify the wrong driver for the data files of your module, you will get undefined behaviour. We could probably do a better job adding sanity checks to be sure the data we retrieve from a module is likely valid values, but these checks would be at the lowest level of engine use and be called very frequently, resulting is slower results all to handle a misconfiguration, so I'm not rushing to add these checks.

The bottom line is, give the LZSS drivers LZSS data and give the Zip drivers Zip data, etc.

Happy Sunday!

Troy

On February 28, 2021 1:45:12 AM MST, David Haslam <dfhdfh at protonmail.com> wrote:
>Still unanswered is Bastian’s good question “Why is it the default
>then?”
>
>David
>
>Sent from ProtonMail Mobile
>
>On Sun, Feb 28, 2021 at 03:06, Troy A. Griffitts <scribe at crosswire.org>
>wrote:
>
>> While I agree we probably shouldn't segfault, this is a bit like
>> renaming xyz.exe to xyz.doc and trying to open it in Word.
>>
>> Not that is matters, but the details are that the LZSS decompressor
>> treats the entire block as uncompressed binary (because it doesn't
>> encounter any compression block signatures, because the data is not
>LZSS
>> data but instead ZIP encoded data) and does nothing with any of it,
>> returning the zip compressed data as-is, resulting in binary
>gibberish
>> when we reference it, so when we pull offset and size values from the
>> "uncompressed" block (which really isn't uncompressed), we get
>> essentially random numbers... which causes things to crash. Not sure
>> what Word does when you give it a ".doc" file which is a binary, or
>> MySQL does when you cat /dev/random > /var/data/mysql/ibdata1, but I
>> would bet it doesn't behave very well.
>>
>> I would say the solution to this is don't give the LZSS filters ZIP
>> compressed data. I can try to put some sanity checks in when I read
>the
>> offset, size values from the "uncompressed" data and might be able to
>> prevent a crash.
>>
>> Troy
>>
>> On 2/27/21 10:19 AM, Bastian Germann wrote:
>>> Am 27.02.21 um 17:47 schrieb refdoc at gmx.net:
>>>> Could you put a bug report into JIRA?
>>>
>>> https://tracker.crosswire.org/browse/API-247
>>>
>>>> As such the LZSS code is experimental and should not be relied
>upon.
>>>>
>>>
>>> Why is it the default then? (according to
>>>
>https://wiki.crosswire.org/DevTools:conf_Files#Required_Elements_with_defaults)
>>> _______________________________________________
>>> sword-devel mailing list: sword-devel at crosswire.org
>>> http://crosswire.org/mailman/listinfo/sword-devel
>>> Instructions to unsubscribe/change your settings at above page
>> _______________________________________________
>> sword-devel mailing list: sword-devel at crosswire.org
>> http://crosswire.org/mailman/listinfo/sword-devel
>> Instructions to unsubscribe/change your settings at above page

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://crosswire.org/pipermail/sword-devel/attachments/20210228/9bd47e89/attachment.html>