[sword-devel] Locale - proposal

Fri Jan 3 15:03:10 MST 2014

Troy,
Chris has documented that the locale mechanism should look for 3 things: language, optional country and optional script. The code as it stands works for any two, treating script as country (I didn't look but assume that country can take a 4 character script.).

The standard (I read it at one point but am not sure what its designation is) has the names in one of the following orders:
L
LC
LS
LSC
There is a separator between them but not sure whether _ or -. And the casing of the parts is well defined. L - lower, C - UPPER, S - Title (If I remember correctly).

Here is some Microsoft documentation on it: http://msdn.microsoft.com/en-us/library/windows/desktop/dd373814(v=vs.85).aspx

DM

On Jan 3, 2014, at 2:16 PM, Troy A. Griffitts <scribe at crosswire.org> wrote:

> Hi guys,
> 
> Let me clear up 2 things, and suggest a third.
> 
> 1) SWORD does have fallback locale logic:
> 
> http://crosswire.org/svn/sword/trunk/src/mgr/localemgr.cpp
> (search for setDefaultLocaleName)
> 
> The problem is here:
> 
> if (!getLocale(tmplang)) {
> 
> // then continue to search for a fallback
> 
> }
> 
> getLocale never returns NULL. At some point, we changed it to return the default ("en_US") if a locale was requested which doesn't exist (which obviously makes the "if (!getLocale(...))" check never true).
> 
> This is a bug and should be the check should be changed to:
> 
> if (locales->find(tmplang) == locales->end()) {
> 
> I have just checked in a bug fix (and so the link above with show fixed code, sorry, but you can guess what it used to read).
> 
> 
> 2) If you'd like to display locale names to the user, please consider using the meta locale "locale" (file:locale.conf), which has an exhaustive list of all known locales along with their English counterparts. This can be used like this:
> 
> LocaleMgr *lm = LocaleMgr::getSystemLocaleMgr();
> std::cout << lm->translate("de", "locales") << "\n"; // prints "Deutsch"
> std::cout << lm->translate("de.en", "locales") << "\n"; // prints "German"
> 
> Peter, you may wish to grab the SystemLocaleManager as I've done above rather than using your previously posted:
> 
> LocaleMgr *localeMgr = new LocaleMgr();
> 
> This will save you scanning all the folders and reloading all the discovered locales.
> 
> But we seem to have an inconsistency here. In our locales.d/ files we use, e.g.,
> 
> Name=zh_Hant
> 
> In the locales.conf we use:
> 
> zh-Hant=繁体中文
> 
> (note the _ vs. -)
> 
> We also have many inconsistencies between our locale names and what is in the exhaustive locales.conf file.
> 
> 
> 3) Yeah, I agree with both of you that: a) if we have a specific locale but no general parent, the specific should be copied to the parent until such time as we get a better parent; b) we should not duplicate the other way round general to specific, as the fallback mechanism will cover this.
> 
> And finally, we need to clean this stuff up and make it consistent.
> 
> Chris, what would you suggest to use between underscore and dash, e.g., "zh_Hant" or "zh-Hant"?
> 
> 
> Remember, we have MANY MORE language modules than we have locales for the engine. Wycliffe alone has made sure of this. The locales.conf file is intended to help frontend developers display language names to end users when they come across a module language code.
> 
> Having said this, using it for looking up a display name for a language from our locales should be an acceptable use as well, as suggested here.
> 
> We just need to be sure we are consistent between:
> 
> module.conf: Lang=zh_Hant
> locale.conf Name=zh_Hant
> locales.conf: zh_Hant=
> 
> 
> Troy
> 
> 
> 
> On 01/03/2014 11:10 AM, DM Smith wrote:
>> 
> > On Jan 3, 2014, at 11:37 AM, Peter von Kaehne <refdoc at gmx.net>
> > wrote:
> >
> >>> On Fri, 2014-01-03 at 10:54 -0500, DM Smith wrote: BTW, I like
> >>> how Java searches for localized resource files. The actual
> >>> implementation is rather complex (because it searches multiple
> >>> locations), but to simplify: Given a language code, a country
> >>> code and a script code (script is new to Java 7), it looks for
> >>> the most specific first (i.e. using all three) and then looks for
> >>> a bit less specific (i.e. lang/country and lang/script) and then
> >>> for least specific (i.e. lang). Failing that it returns the
> >>> application default, which does not specify any language, country
> >>> or script.
> >>>
> >>
> >> The C++ sword engine does not do the search for parents as you
> >> explain for Java.
> >
> > Right SWORD looks for exactly what is requested, nothing more. I was
> > suggesting a change to the engine.
> >
> > Without an engine change you have to have file duplication.
> >
> >
> >>
> >> E.g. there is a locale de. Searching for de_DE should bring up de
> >> in absence of de_DE, but this does not happen. I have checked
> >> that.
> >
> > In Java looking for de would never find de_DE. For this, your
> > suggestion of renaming the file is needed. de_DE -> de.
> >
> > However, if a SWORD program only looks for de, it won't find de_DE.
> > The SWORD engine has no fallback.
> >
> > Your suggestion was that a search for de to find de_DE. This goes
> > from the less specific to the more specific and I don't think that
> > makes sense. And it may come up with two choices as with Portuguese.
> >
> >>
> >> But even if we make the engine's search more intelligent (to search
> >> for "parent" if the "offspring" search has failed), we still need
> >> to create duplicate locales for some of the situations where we
> >> have no "parent" locale, just "offspring" -e.g. in Russian and
> >> Arabic.
> >
> > Yes. In the case of JSword, if we only have one choice, Egyptian
> > Arabic, we make that the default, ar. If we know it should not be the
> > default, we also duplicate it as ar_EG. When we get another version
> > of Arabic, we create ar_XX, or rename ar to ar_EG and create ar since
> > XX is a better default.
> >
> > And if you have no fallback mechanism, then you have to do it via
> > file duplication.
> >
> >> Chinese is complicated and I do not know what the right solution
> >> is. Maybe in some places failure is the right answer.
> >
> > For Chinese JSword assumes that \ in mainland China that Simplified
> > is used and Traditional elsewhere.
> >
> > The proper way is to support script and have that work. (Which will
> > be when JSword uses Java 7).
> >
> > So rather than zh_TW and zh_CN you'd have zh_hant and zh_hans.
> >
> >>
> >> Peter
> >>
> >>
> >>
> >>
> >>
> >> _______________________________________________ sword-devel mailing
> >> list: sword-devel at crosswire.org
> >> http://www.crosswire.org/mailman/listinfo/sword-devel Instructions
> >> to unsubscribe/change your settings at above page
> >
> >
> >
> > _______________________________________________ sword-devel mailing
> > list: sword-devel at crosswire.org
> > http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to
> > unsubscribe/change your settings at above page
> 
> 
> 
> _______________________________________________
> sword-devel mailing list: sword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4145 bytes
Desc: not available
URL: <http://www.crosswire.org/pipermail/sword-devel/attachments/20140103/fa8abdf1/attachment-0001.p7s>