From: Philo, John <jphilo@amgen.com>
  To  : 'rasmb' <rasmb@bbri.harvard.edu>
  Date: Wed, 9 Apr 1997 11:14:20 -0700

binary data: the cross-platform problem

One of the reasons I opened this "can of worms" was the fact that I had
heard that there was consideration by Beckman of switching to a binary
format for primary data storage, and I feared this might potentially
limit the development of software by others and on other platforms.

Let me say that I am a VERY strong believer in the importance of having
many different ways to analyze the data, and that we should not be
dependent on whatever software Beckman provides, and thus that we do not
want to doing anything that might prevent or limit such development.

Therefore yesterday Walt Stafford and I did some investigating into the
question I originally posed: would it be possible to define a binary
format that would work across platforms? (In this case, Mac versus PC).

In short, I think my fears were well-founded.  It would be very, very
difficult for Mac users to read binary formats defined on the PC, and
probably impossible in certain programming languages, and I believe
similar problems would arise for DEC and Unix machines.

For those interested, the details are below.

John Philo

-----------------

Walt Stafford sent me some of the binary data his new DCDT program uses
and the details of the format.  His binary format cleverly and compactly
 encodes values for (10000 * the fractional fringe displacement) from
interference scans as a 16-bit unsigned integer, i.e. a type of data
that one would think would be the MOST compatible across platforms, and
a data type which certainly avoids the potential nightmare of different
floating-point formats.  

However, it turns out that the 16 bit integers in his files have the
most-significant (hi) and least-significant (low) bytes stored in the
opposite order from how they are conventionally stored on PC's.
(Actually I think it is the order on the PC that should probably be
regarded as "backwards").  

Anyway, what this means, for example, is that if the Beckman data
acquisition generated binary files, then programs on Macs (and I think
Unix and DEC too) would have to read those data byte-by-byte, multiply
the hi byte by 256, and add the low byte.  Yes, this could be done in
SOME languages, but certainly not all, and at best it would take away
much of the speed advantage of binary files.