[jsword-devel] Idea for optimisation/improvement in iso639 lang props init

Wed Dec 29 10:40:36 MST 2010

Hi DM,

I like your idea of lazy-loading parts of the conf files but see the last
point for a simpler thing you may like to try first.

Regarding your comment '*the default resource bundle should not be loaded by
the VM if it is not needed*' can I check you are not confused by the same
issue that confused me for a few days.  I did not understand why the default
resource bundle was being loaded as well as the English (for me) one.  Then
I realised that the English bundle modifies the default one rather than
overriding it so both bundles need to be loaded.  So actually the default
bundle will always be loaded.

There is another related issue I have known about for some time but have not
mentioned but could speed up the initial loading of conf files.  When
starting And Bible I get an awful lot of the following messages which I
think might be caused by the BufferedReader used to load conf files but I
haven't analysed this in detail so it could be any BufferedReader used in
JSword.   I get the message 22 times on startup and have 10 modules.
     Default buffer size used in BufferedReader constructor. It would be
better to be explicit if an 8k-char buffer is required.
Maybe the BufferedReader used to load conf files could be specified to be
initially 2k and any other BufferedReaders could also use an appropriate
setting.

Best regards
Martin

On 29 December 2010 16:53, DM Smith <dmsmith at crosswire.org> wrote:

>  We've tried a variation of this. The only difference is that we don't have
> localizations of the names in the default property files. SWORD and the
> frontends, such as Xiphos, do not have mappings of codes to names. Xiphos in
> particular uses a 3rd party lookup to get them.
>
> Regarding our lookup:
> The default list is a master list of all languages. I update it about once
> or twice a year from the official source. It is only in English. At one time
> we used a subset of this based upon what was in the CrossWire repository.
> The lookup mechanism needs a default or for the software to do something
> reasonable when the lookup fails. If you were to replace the default with
> the smaller _en version, you'd get the "Other" name for the unknown code.
> (This is the subset that we used.)
>
> Why did we move away from that?
> There was a flurry of new module creation.
> * CrossWire got a lot of modules for "minority" languages. Many of these
> people groups are small and impoverished. The primary users of these modules
> would be pastors, missionaries and perhaps scholars.
> * CrossWire got some modules for languages that are well known to many.
> E.g. Latin, Old English.
> * CrossWire got a bunch of modules for language groups that we wanted to
> reach. E.g. Farsi.
> * The Xiphos module repository came into its own. Unlike the CrossWire
> repository it is updated frequently.
>
> The basic issue can be summarized:
> How would you feel if your primary language was Farsi and you were looking
> for Farsi modules? Would you mind finding them in "Other" or "Unknown"?
>
> So, I think we should solve these problems:
> Slow startup.
> There will be several reasons for this. But with regard to this issue it
> stems from the confs being fully read and converted into an internal format,
> involving a lookup of language codes to get language names. This process
> should be made to be "lazy."
>
> Slow lookup of new language names.
> I was under the impression that the default resource bundle should not be
> loaded by the VM if it is not needed. We should verify whether this is the
> case or not.
> If we were to move the list of all languages into a different bundle or
> perhaps an entirely different lookup mechanism, that might help.
> I think the localized language names would be a better default. (See:
> http://crosswire.org/wiki/Localized_Language_Names ) This might be better
> for all locales.
>
> Thoughts?
>
> In Him,
> DM
>
>
>
> On 12/29/2010 09:02 AM, Martin Denham wrote:
>
> Hi,
>
>  David's recent comments on the Sword mailing list seem relevant.  They
> might imply that a more limited list of languages might be acceptable and he
> also mentions the approach taken by Xiphos.  He says:
>
> "Referring to http://www.crosswire.org/wiki/DevTools:confFiles
> the only language related element is lang itself.
> This means that when a front-end encounters a new module for which it
> cannot
> yet provide a look-up for the localized language name, it will show Unknown
> as the name of the language. That's what Xiphos does, for example.
> As a fall-back position, might it be sensible to allow the conf file to
> optionally include the localized language name in addition to the lang
> element?
> This would be processed as follows:
> If the lang identifier can be looked up, use the look-up value.
> If not, then if the conf file includes the localized language name, use
> that.
> Then if not either, fall back to using unknown, as at present."
>
>
>  Regards
> Martin
>
> On 22 December 2010 22:42, Martin Denham <mjdenham at gmail.com> wrote:
>
>> I was wondering if we need the iso639 properties files if the language
>> codes and names returned by the standard Locale class contained all the
>> codes and names that are in the large properties file - I don't know if they
>> do.  Using Locale could (I think) also automatically give us locale specific
>> country names automatically for all Locales e.g. Angličtina instead of
>> English on Czech mobiles.
>>
>>  I haven't tried this out yet and we are only talking about 2 secs at
>> startup but I am trying to whittle away the 17 secs it currently takes to
>> start up JSword-And Bible.  However, this does possibly give other
>> advantages.
>>
>>
>> On 22 December 2010 18:04, DM Smith <dmsmith at crosswire.org> wrote:
>>
>>> I've gone back and forth on this one. The goal is to have names for all
>>> the codes. There are a bunch of minority languages with Bibles in beta that
>>> this covers.
>>>
>>> The simplest solution is to rename the _en file to be the default.
>>>
>>> The bigger problem is that the confs are fully read on startup which is
>>> in necessary. This is causing the Lang lookup.
>>>
>>> In Him,
>>> DM
>>>
>>> Cent from my fone so theer mite be tipos. ;)
>>>
>>> On Dec 22, 2010, at 7:07 AM, Martin Denham <mjdenham at gmail.com> wrote:
>>>
>>> > I have occasionally tried to improve the slow start-up times of And
>>> Bible & JSword without much success.  However, one area which I have a
>>> suspicion may take time is property file loading.
>>> >
>>> > I run out of memory part way through when profiling startup but I
>>> notice iso639.properties which has 7600 lines can take 2 seconds to load up
>>> on slow phones.  I wonder if, instead of storing the language codes in
>>> properties files we could use methods like Locale.getDisplayLanguage(loc)
>>> and other Locale methods to completely remove the need for the iso639
>>> properties files.
>>> >
>>> > What do you think?
>>> >
>>> > Regards
>>> > Martin
>>> >
>>>   > _______________________________________________
>>> > jsword-devel mailing list
>>> > jsword-devel at crosswire.org
>>> > http://www.crosswire.org/mailman/listinfo/jsword-devel
>>>
>>> _______________________________________________
>>> jsword-devel mailing list
>>> jsword-devel at crosswire.org
>>> http://www.crosswire.org/mailman/listinfo/jsword-devel
>>>
>>
>>
>
> _______________________________________________
> jsword-devel mailing listjsword-devel at crosswire.orghttp://www.crosswire.org/mailman/listinfo/jsword-devel
>
>
>
> _______________________________________________
> jsword-devel mailing list
> jsword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/jsword-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.crosswire.org/pipermail/jsword-devel/attachments/20101229/d74764b2/attachment.html>