[sword-devel] Rendering issues with Finnish Umlauts in FinPR
Tobias Klein
contact at tklein.info
Mon Apr 3 14:18:29 EDT 2023
Thanks Troy!
I'll have a look at the EncodingFilters.
I think this is something not fully clear from the SWORD
documentation/examples.
Maybe these transformation points you had mentioned in the thread below
should be described somewhere in the developer wiki?
Best regards,
Tobias
On 4/3/23 6:45 PM, Troy A. Griffitts wrote:
> Dear Tobias,
>
> Please be sure to note my comment to you below in this thread. It is
> likely the cause of your rendering issues, while other apps have no
> problems.
>
> In brief, it says that I haven't seen anywhere that you tell SWORD
> what markup and encoding you want from the engine. If this is the case
> you will get whatever the modules are encoded / marked up as, which
> might be various things.
>
> Hope this helps,
>
> Troy
>
> On January 22, 2023 12:03:22 PM MST, "Troy A. Griffitts"
> <scribe at crosswire.org> wrote:
>
> Hey guys,
>
> Sorry for not jumping in on this thread more quickly.
>
> Please remember, SWORD has 4 transformation points, each moving
> from the module source (as described in the .conf file) to the
> client's request:
>
> RenderFilters - markup, e.g., GBF, ThML, OSIS -> XHTML
>
> StripFilters - prep before searching
>
> OptionFilters - turning on an off markup in the text stream based
> on user options, e.g., Strongs Number, Words of Christ in Red, etc.
>
> EncodingFilters - e.g., 8859 - > UTF-8
>
>
> Module team: be sure the module has the correct Encoding value in
> the .conf file (or the default)
>
> Tobias, be sure you are creating your SWMgr with the correct
> MarkupFilterMgr to do the transformation you desire, e.g., see:
>
> https://crosswire.org/svn/sword/trunk/examples/cmdline/outrender.cpp
>
> Hope this helps,
>
> Troy
>
>
> On 1/22/23 10:39, Fr Cyrille wrote:
>> HI David,
>> If you send me the file, I can convert it quickly in osis. I
>> script it from imp to usfm and the with u2o.py.
>>
>> Le 22/01/2023 à 16:54, David Haslam a écrit :
>>> Thanks Tobias,
>>>
>>> The problem is that CrossWire no longer accepts module
>>> submissions that use IMP format for the build process.
>>>
>>> We’d need to have a script (or equivalent TextPipe filter) to
>>> convert IMP to OSIS (whether directly or indirectly through some
>>> other intermediate file format).
>>>
>>> I’m not currently in a practical position to work on that kind
>>> of task.
>>> Is anyone else up to it?
>>>
>>> Best regards,
>>>
>>> David
>>>
>>> Sent from Proton Mail for iOS
>>>
>>>
>>> On Sun, Jan 22, 2023 at 15:39, Tobias Klein
>>> <contact at tklein.info> wrote:
>>>>
>>>> The FinPR module that David sent me works fine without
>>>> rendering issues! (see screenshot below)
>>>>
>>>> It would be good to upgrade the module in the repo accordingly.
>>>>
>>>> Best regards,
>>>> Tobias
>>>>
>>>> On 1/22/23 8:31 AM, David Haslam wrote:
>>>>> Thanks Kristóf.
>>>>>
>>>>> The rendering problem could have been fixed a decade ago!!!
>>>>>
>>>>> Checking through my email archives yesterday, I discovered
>>>>> that I had rebuilt the FinPR module exactly 10 years ago! That
>>>>> rebuild used mod2imp and imp2vs and included a fix to the text
>>>>> encoding implemented on the IMP textfile). The message was
>>>>> sent to the modules address on 2013-01-21 but presumably never
>>>>> progressed by Chris Little who was then still supposed to be
>>>>> responsible for module releases and updates. He went
>>>>> permanently AWOL from CrossWire around that time.
>>>>>
>>>>> Back then we had not narrowed the policy for submitted source
>>>>> text to be OSIS XML only.
>>>>>
>>>>> I wrote privately to Tobias last night, forwarding the email
>>>>> of 10 years ago complete with both attachments. He will
>>>>> examine those today.
>>>>>
>>>>> Aside: I also replaced <…> by {…} where these had wrapped the
>>>>> ch:vs references that recorded av11n in the original upstream
>>>>> source. In 2012, there had been no suitable av11n available in
>>>>> SWORD but which we do have more recently.
>>>>>
>>>>> mod2osis should not be used, as has already been noted.
>>>>> A round trip with mod2osis and osis2mod is not lossless,
>>>>> unlike one with mod2imp and imp2vs.
>>>>>
>>>>>
>>>>> Best regards,
>>>>>
>>>>> David
>>>>>
>>>>> Sent from Proton Mail for iOS
>>>>>
>>>>>
>>>>> On Sat, Jan 21, 2023 at 23:15, Kristof Szabo
>>>>> <kristof.szabo at web.de> wrote:
>>>>>> I managed to get Ezra running (it was some libicu70 mess),
>>>>>> and yes, the accented characters in this module are broken
>>>>>> (as other modules accented characters are OK; I assume it is
>>>>>> not a font issue). I tried the conf file change, but it
>>>>>> didn't work either.
>>>>>>
>>>>>> The mitigation was to rebuild the module, mod2osis leaves
>>>>>> some garbage in the OSIS, but that would be easy to clean,
>>>>>> anyway osis2mod is possible with this garbage left in and
>>>>>> tada we have a proper accents.
>>>>>>
>>>>>> image.png
>>>>>>
>>>>>> As the module was updated last only 3,5 yrs ago I assume the
>>>>>> maintainer is still active, ie. they can be reached.
>>>>>>
>>>>>> Or I can have a look too, the challenge is, that such a
>>>>>> module rebuild can open pandora's box, if I run some tests
>>>>>> (https://github.com/krisek/sword-test) or David checks them,
>>>>>> then for sure there will be some issues. I'm happy to fix
>>>>>> some of them, but I definitely do not speak Finnish, so I'm
>>>>>> not sure this would be a responsible action. If Dom gives me
>>>>>> the go I can fix syntax & submit, but I don't want to end up
>>>>>> in the rabbit hole :) Best would be to reach out to the
>>>>>> original maintainer.
>>>>>>
>>>>>> Kind regards,
>>>>>> k-
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Sat, Jan 21, 2023 at 8:26 PM Greg Hellings
>>>>>> <greg.hellings at gmail.com> wrote:
>>>>>>
>>>>>> Is Ezra properly setting encoding on the content it
>>>>>> renders? Is it maybe setting a font that doesn't have the
>>>>>> proper code points?
>>>>>>
>>>>>> --Greg
>>>>>>
>>>>>> On Sat, Jan 21, 2023, 13:12 Tobias Klein
>>>>>> <contact at tklein.info> wrote:
>>>>>>
>>>>>> Hi Kristof, David,
>>>>>>
>>>>>> Adding Encoding=UTF-8 to the module conf file
>>>>>> ~/.sword/mods.d/finpr.conf does not solve my issue.
>>>>>>
>>>>>> The text still looks the same as before ...
>>>>>>
>>>>>> What else could I do to further debug this?
>>>>>>
>>>>>> Best regards,
>>>>>> Tobias
>>>>>>
>>>>>> On 1/21/23 5:18 PM, Kristof Szabo wrote:
>>>>>>> Hi Thomas,
>>>>>>>
>>>>>>> I suppose the problem is that finpr.conf contains no
>>>>>>> encoding information (check the Hun* modules for
>>>>>>> reference), and if there is nothing specified
>>>>>>> Latin-1 is the default. mod2osis (shouldn't be used
>>>>>>> !! :)) shows that the module is in UTF-8, so there
>>>>>>> is a misalignment.
>>>>>>>
>>>>>>> https://wiki.crosswire.org/DevTools:conf_Files#:~:text=Plaintext-,Encoding,-UTF%2D8%0AUTF
>>>>>>>
>>>>>>> Kind regards,
>>>>>>> Kristof
>>>>>>>
>>>>>>> On Sat, Jan 21, 2023 at 4:49 PM David Haslam
>>>>>>> <dfhdfh at protonmail.com> wrote:
>>>>>>>
>>>>>>> Hi Thomas,
>>>>>>>
>>>>>>> What about other Finnish modules?
>>>>>>> eg. FinPR92, FinRK, FinSTLK2017
>>>>>>>
>>>>>>> Presumably you already tested (eg) German
>>>>>>> modules and found that umlauts and eszett are
>>>>>>> both rendered aright?
>>>>>>>
>>>>>>> Btw. FinPR renders aright in PocketSword
>>>>>>> (iOS/iPadOS).
>>>>>>>
>>>>>>> David
>>>>>>>
>>>>>>> Sent from Proton Mail for iOS
>>>>>>>
>>>>>>>
>>>>>>> On Sat, Jan 21, 2023 at 15:25, Tobias Klein
>>>>>>> <contact at tklein.info> wrote:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> When retrieving the text of the FinPR module I
>>>>>>>> am getting some rendering issues with the
>>>>>>>> Finnish Umlauts. This is based on a user's
>>>>>>>> problem report.
>>>>>>>>
>>>>>>>>
>>>>>>>> Romans 5:8 returns like this in
>>>>>>>> node-sword-interface / Ezra:
>>>>>>>>
>>>>>>>> Mutta Jumala osoittaa rakkautensa meit� kohtaan
>>>>>>>> siin�, ett� Kristus, kun me viel� olimme
>>>>>>>> syntisi�, kuoli meid�n edest�mme.
>>>>>>>>
>>>>>>>>
>>>>>>>> While it should like like this (rendered text
>>>>>>>> copied from Xiphos):
>>>>>>>>
>>>>>>>> Mutta Jumala osoittaa rakkautensa meitä kohtaan
>>>>>>>> siinä, että Kristus, kun me vielä olimme
>>>>>>>> syntisiä, kuoli meidän edestämme.
>>>>>>>>
>>>>>>>>
>>>>>>>> This occurs both on Linux and macOS (have not
>>>>>>>> tested on Windows yet).
>>>>>>>>
>>>>>>>> Any pointers what could be the root cause? I
>>>>>>>> generally have not observed rendering issues
>>>>>>>> with other modules.
>>>>>>>>
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>> Tobias
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> sword-devel mailing list: sword-devel at crosswire.org
>>>>>>> http://crosswire.org/mailman/listinfo/sword-devel
>>>>>>> Instructions to unsubscribe/change your settings
>>>>>>> at above page
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> sword-devel mailing list:sword-devel at crosswire.org
>>>>>>> http://crosswire.org/mailman/listinfo/sword-devel
>>>>>>> Instructions to unsubscribe/change your settings at above page
>>>>>> _______________________________________________
>>>>>> sword-devel mailing list: sword-devel at crosswire.org
>>>>>> http://crosswire.org/mailman/listinfo/sword-devel
>>>>>> Instructions to unsubscribe/change your settings at
>>>>>> above page
>>>>>>
>>>>>> _______________________________________________
>>>>>> sword-devel mailing list: sword-devel at crosswire.org
>>>>>> http://crosswire.org/mailman/listinfo/sword-devel
>>>>>> Instructions to unsubscribe/change your settings at above
>>>>>> page
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> sword-devel mailing list:sword-devel at crosswire.org
>>>>> http://crosswire.org/mailman/listinfo/sword-devel
>>>>> Instructions to unsubscribe/change your settings at above page
>>>
>>> _______________________________________________
>>> sword-devel mailing list:sword-devel at crosswire.org
>>> http://crosswire.org/mailman/listinfo/sword-devel
>>> Instructions to unsubscribe/change your settings at above page
>>
>>
>> _______________________________________________
>> sword-devel mailing list:sword-devel at crosswire.org
>> http://crosswire.org/mailman/listinfo/sword-devel
>> Instructions to unsubscribe/change your settings at above page
>
> --
> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>
> _______________________________________________
> sword-devel mailing list: sword-devel at crosswire.org
> http://crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://crosswire.org/pipermail/sword-devel/attachments/20230403/2705b7f7/attachment-0001.htm>
More information about the sword-devel
mailing list