[sword-devel] GlobalOptionFilter=UTF8GreekAccents and non-Greek modules

[David Haslam]

These are the principal diacritics found in Biblical Greek that have to be
removed with a UTF8GreekAccents filter.

The first five are general accents, not particular to Greek.
It's on account of these that the filter should not be applied to non-Greek
text.

U+0300	̀	COMBINING GRAVE ACCENT
U+0301	́	COMBINING ACUTE ACCENT
U+0308	̈	COMBINING DIAERESIS
U+0313	̓	COMBINING COMMA ABOVE
U+0314	̔	COMBINING REVERSED COMMA ABOVE
U+0342	͂	COMBINING GREEK PERISPOMENI
U+0343	̓	COMBINING GREEK KORONIS
U+0344	̈́	COMBINING GREEK DIALYTIKA TONOS
U+0345	ͅ	COMBINING GREEK YPOGEGRAMMENI

No other diacritics or characters should be removed. 
Though there are a few more combining accents in this block, they aren't
really used in Biblical Greek.
I am open to correction on this point.

e.g. The right single quotation mark (U+2019) is NOT a diacritic. It should
not be removed.

Before any of these accents can be removed, they must first be separated
from the Greek letters they are combined with. 

Although normalization to the decomposed form can produce this effect, as we
have seen already, this can have undesirable side effects on any non-Greek
text in the module that may happen to include combined or unusual
characters.

It would therefore be more sensible to simply use a comprehensive mapping
table that replaces each possible accented character by the corresponding
letter in the Greek alphabet. In this way the filter can completely avoid
the need to apply any Unicode normalization. 

The complete mapping table would have at least 130 rows. It will need to
take into account that there are at least 75 possible combinations of a
letter with two accents. There are none with three.

Any residual combining characters should also be removed, to cover the
possibility that a module may have been intentionally made without
normalizing the Greek source text by default to NFC.

That's my proposal. I can easily create such a mapping table that
programmers can use.
I can also readily test it with a bespoke TextPipe filter.


Best regards,

David





--
View this message in context: http://sword-dev.350566.n4.nabble.com/GlobalOptionFilter-UTF8GreekAccents-and-non-Greek-modules-tp4656719p4656765.html
Sent from the SWORD Dev mailing list archive at Nabble.com.