[SOLVED][SynEdit] GetNextHighlighterTokenFormView: Token starts with
char
lazarus / 31-LCL

----------------------------------------------------------------------
Pascal · 2015-08-13 05:20

How can i prevent this error when typing multibyte characters?
I use a SynEdit with a modified Cobol-Highlighter

Pascal

----------------------------------------------------------------------
jc99 · 2015-08-13 06:40

First read  http://wiki.lazarus.freepascal.org/Lazarus_Faq#What_is_the
_correct_way_to_ask_questions_in_the_forum.3F
Meaning: What version, what OS, Some Code ...
Maybe a Test-Project, ...
So we can verify your bug, maybe you found a bug in the component, or
it's just not released for those chars.

----------------------------------------------------------------------
Pascal · 2015-08-13 08:40

Sorry!

Version is 1.4.0 on Windows 8.1

----------------------------------------------------------------------
Basile B. · 2015-08-13 10:24

Next time don't forget that SynEdit has its own subforum:
http://forum.lazarus.freepascal.org/index.php/board,69.0.html

----------------------------------------------------------------------
Martin_fr · 2015-08-13 13:43

Quote from: Pascal on August 13, 2015, 07:20:07 amI use a SynEdit with
a modified Cobol-Highlighter

Does it happen with the not modified HL too?

In any case the error message is not worded very helpful, what it
means is, that the highlighter returned a pointer to a token (part of
the line text).
But the pointer points into the middle byte of utf8 codepoint.

In other words the HL tries to split a single char, so that the first
byte(s) of it have one color, and the remaining byte(s) another.

----------------------------------------------------------------------
Pascal · 2015-08-14 06:09

@BBasile: Thanks for the hint. I didn't recognized it so far.

@Martin_fr: Thanks for the explenation. That makes sense as the Cobol-
Highlighter doesn't work with multi-byte-characters.
It steps byte wise thru the line which works if you do not use
äöüÄÖÜß....
Is there a way to step column wise thru the line (line[collumn]) or do
i have to use a char-pointer to handle multi-byte-characters?

Regards
Pascal

----------------------------------------------------------------------
Pascal · 2015-08-14 11:41

I found the solution in TSynPasSyn Pascal Highlighter:

Just added:
Code: [Select]   while (fLine[Run] in [#128..#191]) OR // continued
utf8 subcode
   ((fLine[Run]<>#0) and (fProcTable[fLine[Run]] = @UnknownProc)) do
inc(Run);
to my UnknownProc.

Thanks for the hints.

Regards
Pascal