UTF8 symbols in Editor - strange copy & paste lazarus / 9-Editor ---------------------------------------------------------------------- winni · 2019-07-01 22:52 Hi I just wanted to type a simple string: Code: Pascal  [Select][+][-]window.onload = function(){var x1 = document.getElementById("main_content_section"); if (x1) { var x = document.getElementsByClassName("geshi");for (var i = 0; i  Zodiak = '♈♉♊♋♌♍♎♏♐♑♒♓';  I selected the UTF8 symobls in KCharSelect and copied it into the clipboard. Checking the clipbboard to ensure that it's simple unformated text. I copy the string into the IDE editor - and the symols are not just the simple one I wanted, but some kind of icons: painted on a circle and colored - see attachment. I know that some fonts have now such crazy icons, but * The Editor Font is Liberation Mono - with simple symbols * The Desktop Font is Liberation Sans - with simple symbols So - where do the icons come from? And: is this an IDE issue or (again) a KDE Plasma problem? System: KDE Plasma 5.16,  Suse Tumbleweed Winni ---------------------------------------------------------------------- lucamar · 2019-07-02 00:59 Try to see how those symbols are implemented (if they are) in Liberation Mono--with a charset map application, for example. Just tested and in Liberation Mono they are "normal" symbols. Note, though, that despite how they "look" they are in fact the same characters. What happens if you close and then reopen that file? ETA: It should look like the attached image (Lazarus/Ubuntu + Liberation Mono) ---------------------------------------------------------------------- winni · 2019-07-02 01:20 Quicker than me ... I tested it in Libre Office writer - simple symbols, as I saw it in KCharSelect. Reopening: I also tested - no effect. And now: Opened good old joe in a terminal. Pure utf8. Copied the symbols to the editor. Called it zodiak.inc and opened  that file in the IDE: Colored icons. Think we need an IDE magician to get the miracle solved !!?? Winni ---------------------------------------------------------------------- lucamar · 2019-07-02 01:24 I don't think it's the IDE per se, but something to do with your system. Note that my image shows the string in Lazarus. Unfortunately I can't test with KDE at the moment, I don't have any (working) KDE system at hand here. ---------------------------------------------------------------------- PascalDragon · 2019-07-02 07:14 It's probably related to the font. Some font supports colored glyphs (e.g. Emojis) and if the widgetset backend (Qt/Gtk) supports them they might be used automatically. ---------------------------------------------------------------------- winni · 2019-07-02 22:08 No it's not related to the font as mentioned above. I did the same test with the liberation font on Windows 7 with Lazarus 2 - and everything is ok. And loading the pure utf8 file "zoadiak.inc" (see above) showed the utf8 symbols and not those icon-like signs. So there is a problem with the clipboard of KDE Plasma. I'll have a look what Mr. Google knows. Winni ---------------------------------------------------------------------- engkin · 2019-07-03 16:51 As explained by PascalDragon, two things: 1-The font should include emoji style glyphs, beside the text style glyphs for some Unicode points. (A substitution from another font might be possible) 2-The glyph rendering routine should choose the correct glyph. Text style is chosen for some glyphs by default. While the emoji style is chosen for some other glyphs also by default. The zodiac signs are supposed to be rendered using the emoji style. An application can request the text style by using U+FE0E after the desired Unicode. It is up to the rendering routine to fulfill the request properly. This is part of the Unicode standard, and its implementation might not be up to date in different systems/applications/rendering routines. Notice that the font itself is not enough to replicate the issue. In another word, having a different result when using the same font on Windows is expected. ---------------------------------------------------------------------- lucamar · 2019-07-03 17:47 Quote from: engkin on July 03, 2019, 06:51:43 pmThe zodiac signs are supposed to be rendered using the emoji style. An application can request the text style by using U+FE0E after the desired Unicode. It is up to the rendering routine to fulfill the request properly. This is part of the Unicode standard, [...] Huh? Part of the standard? Where have you seen that? AFAIK, the recommendation is exactly the contrary: don't render "emoji" unless explicitely asked for. Have that changed? And even if it's as you say, I seriously doubt that Liberation (much less the "Mono" font!) have such colorful "emojis" (unless they were added recently), so they have to be coming from another font that the system is substituting, which sounds rather strange to me. But then, I know little of these new-fangled fashions ---------------------------------------------------------------------- engkin · 2019-07-03 18:10 Quote from: lucamar on July 03, 2019, 07:47:21 pmHuh?    Text vs Emoji. Edit: Changed the URL from http*s* to http. ---------------------------------------------------------------------- lucamar · 2019-07-03 18:34 Quote from: engkin on July 03, 2019, 08:10:20 pm   Text vs Emoji. Oh!. OK, thanks ---------------------------------------------------------------------- winni · 2019-07-03 22:21 Me also: Huh???? But when I try to open https://xahlee.info/comp/text_vs_emoji.html then i get an SSL_ERROR_INTERNAL_ERROR_ALERT - in Firefox, Chromium and Vivaldi. Anyway - let's grap the facts: Liberation Fonts have no major updates in the last 10, years - and the last big update was the creation of a narrow font: https://github.com/liberationfonts/liberation-1.7- fonts/blob/liberation-fonts-1_07_3/ChangeLog So there are definitly no Emojis in Liberation Fonts. I never had problems neither with KDE <= 4.x nor with windows with this issue: Even copying formated text appeared in the editor, like it should. And why is the clipboard or the fontmanager or who the hell ever overriding the IDE options about the font? I played around with this problem and detected just an astonishing problem in th GIMP: * Copy the 12 unformated zodiak signs to the clipboard. * Create a textbox with any font you like * Copy  the clipboard into the textbox: The font name  changes at once to "sans-serif" and (!!!) * you cannot change the font style anymore!!!! * Weird! I Think there is a problem with the clipboard of KDE Plasma. But I'm not quite shure. Winni ---------------------------------------------------------------------- engkin · 2019-07-03 22:36 Quote from: winni on July 04, 2019, 12:21:40 amMe also: Huh???? But when I try to open https://xahlee.info/comp/text_vs_emoji.html then i get an SSL_ERROR_INTERNAL_ERROR_ALERT Try it without SSL: http://xahlee.info/comp/text_vs_emoji.html Quote from: winni on July 04, 2019, 12:21:40 amSo there are definitly no Emojis in Liberation Fonts. As I said: Quote1-The font should include emoji style glyphs, beside the text style glyphs for some Unicode points. (A substitution from another font might be possible) Quote from: winni on July 04, 2019, 12:21:40 amI never had problems neither with KDE <= 4.x nor with windows with this issue: Even copying formated text appeared in the editor, like it should. And why is the clipboard or the fontmanager or who the hell ever overriding the IDE options about the font? Newer version trying to comply with the "standard". ---------------------------------------------------------------------- Martin_fr · 2019-07-03 23:05 There is another side that may explain the text vs emoji http://xahlee.info/comp/text_vs_emoji.html SynEdit is not utf8 complete. And it relies currently on its own hardcode info on character properties, rather than getting this from the OS. So the variation markers are not known to it. Which, if they are present, may mean that SynEdit will calculate wrong char positions and/or the selectors will not do their work. Open components\synedit\synedittextbuffer.pp line 920 function TSynEditStringList.LogicPosIsCombining(const AChar: PChar): Boolean; and add this line (2nd to last)    ( (AChar[0] = #$EF) and (AChar[1] = #$B8) and (AChar[2] in [#$80..#$8F]) ) or  // Invisible variation selector ---------------------------------------------------------------------- winni · 2019-07-03 23:25 @engkin: thanx for the URL. But which idiot invented that?? UTF8 war a smooth and round wheel; now it has rags and spikes! The SSL-error might come from the last SSL update 10 days ago: with Tumbleweed you are not allways backward compatible ..... @ Martin_fr: yes, that me be part of the solution of the behaviour of the IDE. But I think thats not everything - see the strange experience with the GIMP. Btw: Libre Office Writer  6.2.4.2.0 doesn't   care about icons anyway - good crew! Winni ---------------------------------------------------------------------- engkin · 2019-07-03 23:42 Quote from: winni on July 04, 2019, 01:25:36 amBut which idiot invented that?? UTF8 war a smooth and round wheel; now it has rags and spikes! LOL Quote from: winni on July 04, 2019, 01:25:36 amBtw: Libre Office Writer  6.2.4.2.0 doesn't   care about icons anyway - good crew! I will not be optimistic. Sooner or later they might follow the "standard". ---------------------------------------------------------------------- winni · 2019-07-06 00:02 @Martin_fr I have added your "magic" line into synedittextbuffer.pp line 902 (not: 920) and compiled everything. But that did not help: same symptoms as before: the zodiak is show as emojis, not as signs. I still think the KDE clipboard is buggy. But a lot of thanx anyway ---------------------------------------------------------------------- engkin · 2019-07-06 00:18 You tried it with the variation marker U+FE0E? Edit: It would help to show the code you used. ---------------------------------------------------------------------- Martin_fr · 2019-07-06 00:50 A bit of googling. Looks like it is a Pango feature. Afaik the LCL-gtk uses pango... There are some references that it can be controlled by editing fontconfig, but I did not test. Also that sounds like a system wide change. Not per APP. Also if it is pango, then there might be an API call to switch it off (but the LCL would not have that). Or maybe some other apps use alternatives to pango? ---------------------------------------------------------------------- winni · 2019-07-06 20:05 Concerning KDE: They hat a problem with disabeling Emojis in Kmail. Nothing else with a quick view found. And now to fontconfig: We are not the only one, who are bothered.  The chaos in detail, have a look a this project: https://github.com/stove- panini/fontconfig-emoji He is realy bugged. Btw: WordPress now has a module to disable Emojis. Would be nice for Lazarus! Winni