But `"türki̇ye".upper()` works correctly, at least in Python 3.9.
Interestingly, `"türki̇ye".title()` does _not_ work correctly, returning `"Türki̇Ye"`, presumably because the "title-case" algorithm incorrectly detects \xcc\x87 as punctuation. Not sure if this has been fixed in 3.10 or 3.11.
Python 3.9.13 (main, May 24 2022, 21:13:51)
[Clang 13.1.6 (clang-1316.0.21.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> s = 'TÜRKİYE'
>>> s.lower()
'türki̇ye'
>>> s.lower().upper()
'TÜRKİYE'
>>> s.lower().title()
'Türki̇Ye'
>>> s.lower().encode()
b't\xc3\xbcrki\xcc\x87ye
Edit: It turns out that this behavior is documented [0], and the more-correct routine is `string.capwords` [1]:
> The algorithm uses a simple language-independent definition of a word as groups of consecutive letters. The definition works in many contexts but it means that apostrophes in contractions and possessives form word boundaries, which may not be the desired result ... The string.capwords() function does not have this problem, as it splits words on spaces only.
Compared this vs another comment and its interesting. Pulled the strings directly from the ISO website. I wonder if the 'i' in the lower case one is supposed to be special and not ASCII?
Python 3.8.10 (default, Mar 15 2022, 12:22:08)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> short_name_lower = "Türkiye"
>>> short_name = "TÜRKİYE"
>>> short_name.encode() ## The Ü and İ are UTF-8 chars
b'T\xc3\x9cRK\xc4\xb0YE'
>>> short_name_lower.encode() ## Note only the ü is special, i is just ASCII
b'T\xc3\xbcrkiye'
>>> short_name.lower()
'türki̇ye'
>>> short_name_lower.lower()
'türkiye'
>>> short_name.lower() == short_name_lower.lower() ## Looks the same, but it isn't
False
>>> short_name.lower().encode() ## The i has extra \xcc\x87 here
b't\xc3\xbcrki\xcc\x87ye'
>>> short_name_lower.lower().encode() ## The i doesn't have the extra here
b't\xc3\xbcrkiye'
>>> short_name_lower.upper().encode() ## So this is wrong too, since its just an ASCII i to start
b'T\xc3\x9cRKIYE'
> In Turkish, the character “i” becomes “İ” when capitalized, while the “ı” (a Turkish-specific character) becomes “I” (which looks just like the Latin upper case “I”).
> The out-of-the-box capitalization method implemented by developers or by localization tools by default is often the standard ‘toUpper()’, which doesn’t follow language-specific rules and will convert the “i” into an “I”. As for the lower case “ı”, it will simply fail to capitalize it at all. This will result in a very strange looking text in the game with uncapitalized characters and wrongly capitalized ones.
On my browsers (both Vivaldi and Safari, on MacOS), your string "türki̇ye" is rendered with two dots over the i (stacked vertically). I don't know if this is what you intended, but it doesn't seem to me as the correct lowercase form. But I'll defer to someone more versed in the local customs.
This might actually be a font thing, depending on whether any given font provides a precomposed version of "i" + "combining dot above" (and rendering it as just "i") or not.
Very interesting. On Windows in Chrome its just an i for me, but in Firefox its showing an extra dot between i̇y. But in my comment I can't get that to show up even in Firefox.
It's not i̇, it's i. Also capwords only capitalizes the first letter in each word. Here it's a bandaid. For locale-aware case conversions in python use ICU:
Interestingly, `"türki̇ye".title()` does _not_ work correctly, returning `"Türki̇Ye"`, presumably because the "title-case" algorithm incorrectly detects \xcc\x87 as punctuation. Not sure if this has been fixed in 3.10 or 3.11.
Edit: It turns out that this behavior is documented [0], and the more-correct routine is `string.capwords` [1]:> The algorithm uses a simple language-independent definition of a word as groups of consecutive letters. The definition works in many contexts but it means that apostrophes in contractions and possessives form word boundaries, which may not be the desired result ... The string.capwords() function does not have this problem, as it splits words on spaces only.
[0]: https://docs.python.org/3/library/stdtypes.html#str.title
[1]: https://docs.python.org/3/library/string.html#string.capword...