Subj : Weekly nodelist report on noteworthy changes (355)
To   : Wilfred van Velzen
From : Michiel van der Vlist
Date : Sat Dec 21 2024 16:44:15

Hello Wilfred,

On Saturday December 21 2024 16:11, you wrote to me:

 MvdV>> File: z2daily.356

 MvdV>> 1542 lines read.
 MvdV>> 1513 lines found with ASCII only.
 MvdV>> 29 lines found with well formed UTF-8 sequences.
 MvdV>> 0 lines found with an ill formed UTF-8 sequence.
 MvdV>> 0 lines found with a BOM.

 WV> I don't think such a test can give you 100% certainty it's all utf8.

Maybe not 100%, but surely more that 99,99%. UTF-8 has a fair amount of reduncancy and all the other encodings used in Fidonet are single byte codes. It is extremely difficult if not imposible to emulate well formed UTF-8 sequences with single byte encoding. A "lone" non ASCII character in any single byte encoding will always generate an error. Same for two equal non ASCII characters in a row.

 WV> But the tests I did on my linux machine gave the same result, so it's
 WV> probably fairly certain! ;-)

The ony lines with no single non ASCII characters and no two the same non ASCII charavers in a row are Greek, Mongolian, Russian  and Ukranian. And those are not single byte encoding, they are too long. So in this case it is definitely 100% UTF-8.


Cheers, Michiel

--- GoldED+/W32-MSVC 1.1.5-b20170303
 * Origin: Nodelist Police Station (2:280/5555)

.