Text 12420, 110 rader
Skriven 2022-02-12 15:09:05 av Tim Schattkowsky (2:240/1120.29)
  Kommentar till text 12416 av Rob Swindell (31435.ftsc_pub)
Ärende: Re: Directly include binary data in messages
====================================================
//Hello Rob,//

on *11.02.22* at *20:40:25* You wrote in Area *FTSC_PUBLIC*
to *Tim Schattkowsky* about *"Directly include binary data in messages"*.

 >> @BIN <filename> <CRC32 in hex> <almost binary data>

 RS> Control paragraphs should begin with ^A<tag>: ftsc.org/docs/fts-4000.001
 RS> So assuming '@' represents Ctrl-A, that would just mean putting a colon
 RS> after "BIN". 

It does :)

 RS> But why "BIN"? Wouldn't "IMAGE" be more approrpriate?

Because the same mechanics might be employed for embedding arbitrary binary
data and might be accompanied by a mechanism for referencing the data in the
text (e.g., by file name).

 RS> What purpose is the filename? 

The same as in MIME and further, if no content-type mechanism is added. In that
case, the file extensions may come into play for identifying the file type.

 RS> Since you're using space-delimeters for this control paragraph, you
 RS> couldn't have spaces in the filename unless you had some special escaping
 RS> or quoting syntax support.

Indeed, could have URL-like encoding.

 RS> I would recommend just eliminating the filename unless it provides some
 RS> important function.

I think it does, e.g., for attaching files.

 RS> Is the CRC32 exactly 8 hex digits or not? (i.e. if the first n-hex-digits
 RS> of the CRC value are 0, are they still included?). 

Probably. 

 RS> I assume you mean the IEEE-802.3 CRC-32 algo/polynominal? There are
 RS> multiple 32-bit CRC algorithms, so it would best to be specific.
 RS> en.wikipedia.org/wiki/Cyclic_redundancy_check

Probably.

 RS> "<almost binary data>" is encoding what image format?

Thoug I have the application for images in my mind, this idea is about encoding
arbitrary binary data. The semantics and further mechanics have to be defined
elsewhere.
 
 >> Where <almost binary data> uses a simple encoding that essentially aims at
 >> avoiding $00, $0d and $0a so the resulting string still forms a valid line
 >> of 8-Bit characters. The checksum is also intended to detect any charset
 >> violence or 7-Bit fun that might have happend to the message on the way.
 RS> I would go with base64 encoding to reduce interoperability issues.

Yes and no. We have come a long way in getting rid of the base64 overhead all
across the place (think web services) and it feels somewhat dated to still
stick to it. Just like XML, it is something you basically don't want to have
fresh going into something. 

On the other hand, using base64 without MIME feels like going only half the way
to doing it just the way we do in Emails. That would at least be compatible
with something. However, the main issues I have with this is readability of the
message on legacy systems and the size overhead.

 >> Admittedly, I still have the idea to make the most out of the 63k maximum
 >> message length I spuspect for unsplit messages. W.r.t. this, the above
 >> mechanism actually has the drawback of being incompatible with @SPLIT and
 >> this limiting the size of an attachment effectively to one 63k message.

 RS> Is there really a "63k maximum message length" required by some FTSC

Nope. It has been around in the past (also as a maximum message length
requirement for some echos) simply to save some 16-Bit systems from trouble.
Ich checked my only message base (dating back 25 years and including hundreds
of thousands of messages) and the largest messages are 63k. These are actualy
mostly split longer email messages from an email gateway 15 years ago. However,
63k makes sense at first hand, but surely the whole topic requires testing in
both directions because its pure guesswork.

 RS> standard? SBBSecho definitely doesn't recognize or impose any such limit.

WinPoint surely doesnt bother as long as you stick below 2GB, which is however
probably also the maximum size of the message base file for an area ;)

 >> One main benefit here is the space saving compared to base64. THis is a
 >> general thing: Do you think base64 (or the like) are still necessary or is
 >> betting on 8-bit binary transmission ok?

 RS> I could think of all kinds of character sequences that could cause issues
 RS> with existing software. 

I doubt it. If anything causes trouble in the middle of the line that software
should not exist. Can you give a plausible example?

 RS> These issues would be avoided by just using
 RS> base64-encoding. yEnc is another encoding that's more efficient than
 RS> base64, but I would just stick with base64 as it's just going to avoid a
 RS> lot of potential issues from the outset.

Doing anything beyond base64 actually eventually means to go beyond
compatibility anyway.

Regards,
Tim
--- WinPoint 399.0
 * Origin: Original WinPoint Origin! (2:240/1120.29)