Text 16866, 73 rader
Skriven 2013-11-24 00:46:11 av Maurice Kinal (1:261/38.9)
  Kommentar till text 16861 av mark lewis (1:3634/12.71)
Ärende: utf-8 fidonet messaging
===============================
Hey mark!

 ml> mail processors have no reason to be looking at the content of a
 ml> message

Then how do they know what AREA to toss it to?  Isn't that particular
infomation the very first bytes of the message body (ie content of a message)?

 ml> only the headers and some control lines

The control lines are part of the message body from what I see in the raw pkts
here.  Granted that is probably a lousy place for them but that is where they
are.

In the situation here, I keep the original pkt intact and only process the
vital information extracted to a temporary file which doesn't contain all the
information that is maintained (unaltered) in the original pkt.  In the
temporary file there is absolutely nothing from the pkt header and only the
ascii data (date, to, from, subject) from the msg header(s).  Am I missing
anything vital?  I still see most of the control lines but have started to
strip them out of the temporary file when I actually view them, such as PATH
and SEEN-BY.  So far I haven't seen anything I would use them for being an
enduser, which happens to be my target audience.

 ml> what are you smoking over there?

Nothing at the moment.  However I am aware there could be something worth
smoking in an attempt to figure out what a mail processor is *not* doing as you
seem to be implying.  Please feel free to enlighten if you are aware of a
smokable material that might help illuminate what a mail processor's job should
entail other than what it isn't supposed to be doing.  At the moment I am not
getting it at all.  :-/

Anyhow I do not "catch" or sideline any mail but am aware of at least one mail
processor that will sideline mail if it contains any non-ascii and non-utf8
characters within a message flying the 'CHRS: UTF-8 4' kludge.  I am not
positive how sensitive it is to the '4' level part of the CHRS kludge.

 ml> that is so twisted i can't even follow it

Simple.  0x94 isn't a valid utf-8 code.  Also it isn't an ascii code, which
range from 0x00 to 0x7f.  As for your sent 0xc3 0xb6 combination, the
conversion process assumes they are two non-ascii characters instead of one
utf-8 character which your editor is identifying as 'CHRS: IBMPC 2'.  Those
particular CP437 characters correspond to line/box drawing characters.  However
0xc3 .AND. 0xb6, which is 0x0f6 in utf8-speak, is equivalent to 0x94 in CP437. 
That is why I asked for your confirmation.

Seems pretty clear to me especially without smoking material.  ;-)

 ml> i sent the first byte sequence as a test to see if other software
 ml> would specifically mistake it for a multibyte UTF-8 character

It would have if you hadn't included the following 0x94 character.  Here is a
nice simple test;

------------------- commandline magic starts
echo -e '\xc3\xb6' | file -b -
echo -e '\xc3\xb6 \x94' | file -b -
------------------- commandline magic ends

I get a kick from the output of the second command.  Stinkin' DOS-think. 
Anyhow the non-iso part is right since if you look on any of the 8 bit iso
character tables 0x94 is a control code.  Same with 0x094 in utf-8.  Sending it
in messages will definetly tick off many people using iso codepages which are
many.  I am guessing WAY more than that use CP437.  ;-)

Life is good,
Maurice

... A Møøse once bit my sister ...
--- GNU bash, version 4.2.45(2)-release (x86_64-unknown-linux-gnu)
 * Origin: Pointy Stick Society - Ladysmith BC, Canada (1:261/38.9)