Subj : your external editor test of utf-8
To   : Michiel van der Vlist
From : Maurice Kinal
Date : Mon Apr 21 2014 20:14:03

Hey Michiel!

What follows was originally a reply to the above subject to your message "MSGID: 2:280/5555 535422dd" which you posted to me.  For some strange reason it seems to have dropped off somewhere and I thought I'd try this instead to see if there is a problem at this end.  Offhand I don't think so but if there is then this should tell the tale.  Apologies in advance if you already got this reply and if you do then please let me know.

 MvdV> So you actually reïnvented the wheel and wrote a complete packet
 MvdV> viewer?

Nope.  I just more or less followed the Unix philosophy as I learned it back in the early 1990's.  A healthy part of the philosophy is that unless you think you can do better, use what is presented to you as part of the system.  So the reader is simply a shell wrapper to;

tr '\0\r\1' '\n\n@' < $HOME/archives/new_messages.raw | less

Works like a charm and as long as I don't pipe it to 'less -r' then all is revealed including the BOM when you decide to include it.  With 'less -r' it would be ignored.  Also 'less -r' works great with ansi codes for colourization and the such.

 MvdV> It is a text editor, so you have to apply it to some "text".

Yes including utf-8 characters such as the Euro symbol.  I can also shell out commands and have the output redirected into a message such as this command which should result in the Euro symbol being inserted below;

command:  r ! echo -e "\u20ac"

€

Neat eh!  A better example might be the output of 'lscpu' which will post the specs of the machine I am typing this reply on;

----- output of 'lscpu' starts

Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                4
On-line CPU(s) list:   0-3
Thread(s) per core:    2
Core(s) per socket:    2
Socket(s):             1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 28
Model name:            Intel(R) Atom(TM) CPU D525   @ 1.80GHz
Stepping:              10
CPU MHz:               1800.070
BogoMIPS:              3600.14
L1d cache:             24K
L1i cache:             32K
L2 cache:              512K

----- output of 'lscpu' ends

 MvdV> why reïnvent the wheel?

I didn't.  I use what is presented to me.  All I had to do to make it work with fidonet messaging is was to strip out all the binary from raw pkts and dump the result to a so-called stored msg file although it isn't exactly the same as the ftn stored MSG format it is very simular in structure which makes it easy to redirect it to a outbound pkt such as this reply when I am done creating it.

 MvdV> but there is no need for everyone to do that is there?

Nope.  However I enjoy working with tools that I am comfortable with and that are extremely powerful.  I have yet to find anything comparible in fidonet.  Have you?

BTW the msg scanner part shows your message as: text/plain; charset=utf-8

I don't see a BOM in this one but even if there was one the result will be the same.  However 'iconv' will fail unless the BOM is stipped beforehand.  Just thought you might want to know.

Life is good,
Maurice

.... Don't cry for me I have vi.
--- GNU bash, version 4.3.11(1)-release (x86_64-unknown-linux-gnu)
 * Origin: Pointy Stick Society - Ladysmith BC, Canada (1:153/7001.0)

.