[jsword-devel] Idea for optimisation/improvement in iso639 lang props init

Wed Dec 29 13:54:48 MST 2010

Hi DM,

The message is below.  I included it in the previous message but maybe you
didn't see it.  This is the warning I see a lot of times during startup:

       "Default buffer size used in BufferedReader constructor. It would be
better to be explicit if an 8k-char buffer is required."

Regards
Martin

On 29 December 2010 20:41, DM Smith <dmsmith at crosswire.org> wrote:

>  On 12/29/2010 12:40 PM, Martin Denham wrote:
>
> Hi DM,
>
>  I like your idea of lazy-loading parts of the conf files but see the last
> point for a simpler thing you may like to try first.
>
>  Regarding your comment '*the default resource bundle should not be loaded
> by the VM if it is not needed*' can I check you are not confused by the
> same issue that confused me for a few days.  I did not understand why the
> default resource bundle was being loaded as well as the English (for me)
> one.  Then I realised that the English bundle modifies the default one
> rather than overriding it so both bundles need to be loaded.  So actually
> the default bundle will always be loaded.
>
> Then I'll split it into two resources. The full list will only be used if
> the code was not found otherwise. We'll see about a different lookup for the
> full list. (Ideas??) The load time would still be unacceptable for the
> installer's list of available books. So resource bundle/properties is not
> workable.
>
>
>
>
>  There is another related issue I have known about for some time but have
> not mentioned but could speed up the initial loading of conf files.  When
> starting And Bible I get an awful lot of the following messages which I
> think might be caused by the BufferedReader used to load conf files but I
> haven't analysed this in detail so it could be any BufferedReader used in
> JSword.   I get the message 22 times on startup and have 10 modules.
>
>
> What was the message? I'd like to see it.
>
>
>
>       Default buffer size used in BufferedReader constructor. It would be
> better to be explicit if an 8k-char buffer is required.
> Maybe the BufferedReader used to load conf files could be specified to be
> initially 2k and any other BufferedReaders could also use an appropriate
> setting.
>
>
> Ok.
>
> In Him,
>     DM
>
>
>
>  Best regards
> Martin
>
> On 29 December 2010 16:53, DM Smith <dmsmith at crosswire.org> wrote:
>
>>  We've tried a variation of this. The only difference is that we don't
>> have localizations of the names in the default property files. SWORD and the
>> frontends, such as Xiphos, do not have mappings of codes to names. Xiphos in
>> particular uses a 3rd party lookup to get them.
>>
>> Regarding our lookup:
>> The default list is a master list of all languages. I update it about once
>> or twice a year from the official source. It is only in English. At one time
>> we used a subset of this based upon what was in the CrossWire repository.
>> The lookup mechanism needs a default or for the software to do something
>> reasonable when the lookup fails. If you were to replace the default with
>> the smaller _en version, you'd get the "Other" name for the unknown code.
>> (This is the subset that we used.)
>>
>> Why did we move away from that?
>> There was a flurry of new module creation.
>> * CrossWire got a lot of modules for "minority" languages. Many of these
>> people groups are small and impoverished. The primary users of these modules
>> would be pastors, missionaries and perhaps scholars.
>> * CrossWire got some modules for languages that are well known to many.
>> E.g. Latin, Old English.
>> * CrossWire got a bunch of modules for language groups that we wanted to
>> reach. E.g. Farsi.
>> * The Xiphos module repository came into its own. Unlike the CrossWire
>> repository it is updated frequently.
>>
>> The basic issue can be summarized:
>> How would you feel if your primary language was Farsi and you were looking
>> for Farsi modules? Would you mind finding them in "Other" or "Unknown"?
>>
>> So, I think we should solve these problems:
>> Slow startup.
>> There will be several reasons for this. But with regard to this issue it
>> stems from the confs being fully read and converted into an internal format,
>> involving a lookup of language codes to get language names. This process
>> should be made to be "lazy."
>>
>> Slow lookup of new language names.
>> I was under the impression that the default resource bundle should not be
>> loaded by the VM if it is not needed. We should verify whether this is the
>> case or not.
>> If we were to move the list of all languages into a different bundle or
>> perhaps an entirely different lookup mechanism, that might help.
>> I think the localized language names would be a better default. (See:
>> http://crosswire.org/wiki/Localized_Language_Names ) This might be better
>> for all locales.
>>
>> Thoughts?
>>
>> In Him,
>>  DM
>>
>>
>>
>> On 12/29/2010 09:02 AM, Martin Denham wrote:
>>
>> Hi,
>>
>>  David's recent comments on the Sword mailing list seem relevant.  They
>> might imply that a more limited list of languages might be acceptable and he
>> also mentions the approach taken by Xiphos.  He says:
>>
>> "Referring to http://www.crosswire.org/wiki/DevTools:confFiles
>> the only language related element is lang itself.
>> This means that when a front-end encounters a new module for which it
>> cannot
>> yet provide a look-up for the localized language name, it will show
>> Unknown
>> as the name of the language. That's what Xiphos does, for example.
>> As a fall-back position, might it be sensible to allow the conf file to
>> optionally include the localized language name in addition to the lang
>> element?
>> This would be processed as follows:
>> If the lang identifier can be looked up, use the look-up value.
>> If not, then if the conf file includes the localized language name, use
>> that.
>> Then if not either, fall back to using unknown, as at present."
>>
>>
>>  Regards
>> Martin
>>
>> On 22 December 2010 22:42, Martin Denham <mjdenham at gmail.com> wrote:
>>
>>> I was wondering if we need the iso639 properties files if the language
>>> codes and names returned by the standard Locale class contained all the
>>> codes and names that are in the large properties file - I don't know if they
>>> do.  Using Locale could (I think) also automatically give us locale specific
>>> country names automatically for all Locales e.g. Angličtina instead of
>>> English on Czech mobiles.
>>>
>>>  I haven't tried this out yet and we are only talking about 2 secs at
>>> startup but I am trying to whittle away the 17 secs it currently takes to
>>> start up JSword-And Bible.  However, this does possibly give other
>>> advantages.
>>>
>>>
>>> On 22 December 2010 18:04, DM Smith <dmsmith at crosswire.org> wrote:
>>>
>>>> I've gone back and forth on this one. The goal is to have names for all
>>>> the codes. There are a bunch of minority languages with Bibles in beta that
>>>> this covers.
>>>>
>>>> The simplest solution is to rename the _en file to be the default.
>>>>
>>>> The bigger problem is that the confs are fully read on startup which is
>>>> in necessary. This is causing the Lang lookup.
>>>>
>>>> In Him,
>>>> DM
>>>>
>>>> Cent from my fone so theer mite be tipos. ;)
>>>>
>>>> On Dec 22, 2010, at 7:07 AM, Martin Denham <mjdenham at gmail.com> wrote:
>>>>
>>>> > I have occasionally tried to improve the slow start-up times of And
>>>> Bible & JSword without much success.  However, one area which I have a
>>>> suspicion may take time is property file loading.
>>>> >
>>>> > I run out of memory part way through when profiling startup but I
>>>> notice iso639.properties which has 7600 lines can take 2 seconds to load up
>>>> on slow phones.  I wonder if, instead of storing the language codes in
>>>> properties files we could use methods like Locale.getDisplayLanguage(loc)
>>>> and other Locale methods to completely remove the need for the iso639
>>>> properties files.
>>>> >
>>>> > What do you think?
>>>> >
>>>> > Regards
>>>> > Martin
>>>> >
>>>>
>>>
>
> _______________________________________________
> jsword-devel mailing list
> jsword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/jsword-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.crosswire.org/pipermail/jsword-devel/attachments/20101229/acc6db3e/attachment-0001.html>