[sword-devel] GlobalOptionFilter=UTF8GreekAccents

[DM Smith]

Apparently we are talking past each other.

I understand the argument. The elision mark is not an accent. It is a letter character when used between letters but a punctuation character (so a word break character) when at the beginning or end of words. It has been requested of the Unicode consortium to make it a letter character when it follows a Greek letter, but this has not been done. The goal of the request is that double clicking on the Greek word ending with U+2019 would also select the apostrophe in the same way that it does when it is in the middle of a character sequence.

I’m saying if the filter does double duty for both presentation and normalization of search requests, then both have to work well.

That U+2019 is visually similar to U+0027 and an end user will use the keyboard to type U+0027 when U+2019 is required. This will not work as the exact code point has to match.

Over the years, I’ve found that simple code changes often break other parts of the code.

I’ve requested to examine the search code in Xiphos to see if another filter is applied that strips U+2019. If so, then your request probably will work. However, if Xiphos needs to be changed in addition to SWORD lib to accommodate the change, then probably every frontend would need to be changed.

Also, I need to dig into the Lucene index creation and Lucene search to make sure it doesn’t require indexes to be rebuilt.

DM

> On Mar 18, 2025, at 3:21 AM, David Haslam <dfhdfh at protonmail.com> wrote:
> 
> We don't hide U+2019 in any other context.
> It's use in the KJV for possessives does not hinder search!
> 
> Why the obsession with search, when it has no bearing on my semantic argument?
> An elision mark is simply not an accent!!!  Why is this so hard to understand?
> 
> We wouldn't hide U+2019 in French if we saw it used frequently as an elision mark!!!
> Albeit most French modules to date simply use U+0027 (unlike our KJV).
> 
> Best regards,
> 
> David
> 
> Sent with Proton Mail <https://pr.tn/ref/SWXT9A5YZ67G> secure email.
> 
> On Tuesday, March 18th, 2025 at 1:40 AM, DM Smith <dmsmith at crosswire.org> wrote:
>> 
>> 
>>> On Mar 17, 2025, at 5:24 PM, David Haslam <dfhdfh at protonmail.com> wrote:
>>> 
>>> My argument is simple & straightforward.
>> 
>> Your argument is that display and search should have nothing to do with each other.
>> 
>>> 
>>> When you hide diacritics, you ought not to be hiding punctuation marks.
>>> 
>>> Why is this so contentious?
>> 
>> Software changes can have unintended consequences. It needs to be carefully considered.
>> 
>>> 
>>> In what world does how a module displays require that punctuation be hidden?
>> 
>> In a world where the filter is also used for search.
>>> 
>>> A quotation mark is not an accent! 
>>> 
>>> David
>>> 
>>> 
>>> 
>>> On Mon, Mar 17, 2025 at 21:06, DM Smith <dmsmith at crosswire.org <mailto:On Mon, Mar 17, 2025 at 21:06, DM Smith <<a href=>> wrote:
>>>> 
>>>> 
>>>> 
>>>>> On Mar 17, 2025, at 4:01 PM, Karl Kleinpaste <karl at kleinpaste.org> wrote:
>>>>> 
>>>>> That is, if search success depends on the specificity of whether accents or points are enabled, you're probably doing something wrong.
>>>> 
>>>> Absolutely.
>>>> 
>>>> What we are discussing is whether an apostrophe should be stripped out or not. The apostrophe on the keyboard is U+0027. The apostrophe in the Greek is recommended to be U+2019. They look nearly the same. There are a few other Unicode apostrophes that have the same appearance.
>>>> 
>>>> If the apostrophe isn’t stripped out, then the user input and the text have to agree on which of the several it is. There is no way for the end user to know which. I end up copying from the text to make sure I have the right one.
>>>> 
>>>> I believe the same filter that turns on and off the display of accents is used for searching. David is suggesting that the filter is taking out legitimate level 2 quotation marks from the display when it shouldn’t. I’m suggesting that it needs to remove them for the sake of the search.
>>>> 
>>>> DM
>>> _______________________________________________
>>> sword-devel mailing list: sword-devel at crosswire.org
>>> http://crosswire.org/mailman/listinfo/sword-devel
>>> Instructions to unsubscribe/change your settings at above page
>> 
> 
> _______________________________________________
> sword-devel mailing list: sword-devel at crosswire.org
> http://crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://crosswire.org/pipermail/sword-devel/attachments/20250318/111ae030/attachment-0001.htm>