[jsword-devel] Idea for optimisation/improvement in iso639 lang props init
DM Smith
dmsmith at crosswire.org
Wed Dec 29 13:41:55 MST 2010
On 12/29/2010 12:40 PM, Martin Denham wrote:
> Hi DM,
>
> I like your idea of lazy-loading parts of the conf files but see the
> last point for a simpler thing you may like to try first.
>
> Regarding your comment '/the default resource bundle should not be
> loaded by the VM if it is not needed/' can I check you are not
> confused by the same issue that confused me for a few days. I did not
> understand why the default resource bundle was being loaded as well as
> the English (for me) one. Then I realised that the English bundle
> modifies the default one rather than overriding it so both bundles
> need to be loaded. So actually the default bundle will always be loaded.
Then I'll split it into two resources. The full list will only be used
if the code was not found otherwise. We'll see about a different lookup
for the full list. (Ideas??) The load time would still be unacceptable
for the installer's list of available books. So resource
bundle/properties is not workable.
>
> There is another related issue I have known about for some time but
> have not mentioned but could speed up the initial loading of conf
> files. When starting And Bible I get an awful lot of the following
> messages which I think might be caused by the BufferedReader used to
> load conf files but I haven't analysed this in detail so it could be
> any BufferedReader used in JSword. I get the message 22 times on
> startup and have 10 modules.
What was the message? I'd like to see it.
> Default buffer size used in BufferedReader constructor. It would
> be better to be explicit if an 8k-char buffer is required.
> Maybe the BufferedReader used to load conf files could be specified to
> be initially 2k and any other BufferedReaders could also use an
> appropriate setting.
Ok.
In Him,
DM
>
> Best regards
> Martin
>
> On 29 December 2010 16:53, DM Smith <dmsmith at crosswire.org
> <mailto:dmsmith at crosswire.org>> wrote:
>
> We've tried a variation of this. The only difference is that we
> don't have localizations of the names in the default property
> files. SWORD and the frontends, such as Xiphos, do not have
> mappings of codes to names. Xiphos in particular uses a 3rd party
> lookup to get them.
>
> Regarding our lookup:
> The default list is a master list of all languages. I update it
> about once or twice a year from the official source. It is only in
> English. At one time we used a subset of this based upon what was
> in the CrossWire repository.
> The lookup mechanism needs a default or for the software to do
> something reasonable when the lookup fails. If you were to replace
> the default with the smaller _en version, you'd get the "Other"
> name for the unknown code. (This is the subset that we used.)
>
> Why did we move away from that?
> There was a flurry of new module creation.
> * CrossWire got a lot of modules for "minority" languages. Many of
> these people groups are small and impoverished. The primary users
> of these modules would be pastors, missionaries and perhaps scholars.
> * CrossWire got some modules for languages that are well known to
> many. E.g. Latin, Old English.
> * CrossWire got a bunch of modules for language groups that we
> wanted to reach. E.g. Farsi.
> * The Xiphos module repository came into its own. Unlike the
> CrossWire repository it is updated frequently.
>
> The basic issue can be summarized:
> How would you feel if your primary language was Farsi and you were
> looking for Farsi modules? Would you mind finding them in "Other"
> or "Unknown"?
>
> So, I think we should solve these problems:
> Slow startup.
> There will be several reasons for this. But with regard to this
> issue it stems from the confs being fully read and converted into
> an internal format, involving a lookup of language codes to get
> language names. This process should be made to be "lazy."
>
> Slow lookup of new language names.
> I was under the impression that the default resource bundle should
> not be loaded by the VM if it is not needed. We should verify
> whether this is the case or not.
> If we were to move the list of all languages into a different
> bundle or perhaps an entirely different lookup mechanism, that
> might help.
> I think the localized language names would be a better default.
> (See: http://crosswire.org/wiki/Localized_Language_Names ) This
> might be better for all locales.
>
> Thoughts?
>
> In Him,
> DM
>
>
>
> On 12/29/2010 09:02 AM, Martin Denham wrote:
>> Hi,
>>
>> David's recent comments on the Sword mailing list seem relevant.
>> They might imply that a more limited list of languages might be
>> acceptable and he also mentions the approach taken by Xiphos. He
>> says:
>>
>> "Referring to http://www.crosswire.org/wiki/DevTools:confFiles
>> the only language related element is lang itself.
>> This means that when a front-end encounters a new module for
>> which it cannot
>> yet provide a look-up for the localized language name, it
>> will show Unknown
>> as the name of the language. That's what Xiphos does, for
>> example.
>> As a fall-back position, might it be sensible to allow the
>> conf file to
>> optionally include the localized language name in addition to
>> the lang
>> element?
>> This would be processed as follows:
>> If the lang identifier can be looked up, use the look-up value.
>> If not, then if the conf file includes the localized language
>> name, use
>> that.
>> Then if not either, fall back to using unknown, as at present."
>>
>>
>> Regards
>> Martin
>>
>> On 22 December 2010 22:42, Martin Denham <mjdenham at gmail.com
>> <mailto:mjdenham at gmail.com>> wrote:
>>
>> I was wondering if we need the iso639 properties files if the
>> language codes and names returned by the standard Locale
>> class contained all the codes and names that are in the large
>> properties file - I don't know if they do. Using Locale
>> could (I think) also automatically give us locale specific
>> country names automatically for all Locales e.g. Angličtina
>> instead of English on Czech mobiles.
>>
>> I haven't tried this out yet and we are only talking about 2
>> secs at startup but I am trying to whittle away the 17 secs
>> it currently takes to start up JSword-And Bible. However,
>> this does possibly give other advantages.
>>
>>
>> On 22 December 2010 18:04, DM Smith <dmsmith at crosswire.org
>> <mailto:dmsmith at crosswire.org>> wrote:
>>
>> I've gone back and forth on this one. The goal is to have
>> names for all the codes. There are a bunch of minority
>> languages with Bibles in beta that this covers.
>>
>> The simplest solution is to rename the _en file to be the
>> default.
>>
>> The bigger problem is that the confs are fully read on
>> startup which is in necessary. This is causing the Lang
>> lookup.
>>
>> In Him,
>> DM
>>
>> Cent from my fone so theer mite be tipos. ;)
>>
>> On Dec 22, 2010, at 7:07 AM, Martin Denham
>> <mjdenham at gmail.com <mailto:mjdenham at gmail.com>> wrote:
>>
>> > I have occasionally tried to improve the slow start-up
>> times of And Bible & JSword without much success.
>> However, one area which I have a suspicion may take time
>> is property file loading.
>> >
>> > I run out of memory part way through when profiling
>> startup but I notice iso639.properties which has 7600
>> lines can take 2 seconds to load up on slow phones. I
>> wonder if, instead of storing the language codes in
>> properties files we could use methods like
>> Locale.getDisplayLanguage(loc) and other Locale methods
>> to completely remove the need for the iso639 properties
>> files.
>> >
>> > What do you think?
>> >
>> > Regards
>> > Martin
>> >
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.crosswire.org/pipermail/jsword-devel/attachments/20101229/f587bdeb/attachment-0001.html>
More information about the jsword-devel
mailing list