From: Tom Laue <tom.laue@unh.edu>
  To  : demeler@bioc02.uthscsa.edu
  Date: Wed, 09 Apr 1997 13:12:59 -0400

Re: Data storage format

Borries, 
I thought you might be interested in my response to Jo Butler.
Tom

Dear Jo,
Thank you for your comments! You are right about the universality of ASCII
files, and I agree that they should be retained. For those of us working
out at the edge (which will become the norm in just a few years), it has
become obvious that some form of data compression will be needed. We can
easily generate 100 MBytes of data in a single day, and even with very
large disks, the problem of retaining and cataloging all of the files is
overwhelming. With the addition of new detectors, this problem will be
exacerbated.

I believe that the idea of a database remains viable regardless of the
output file format. Conversion within the database would be available.
Perhaps a viable alternative would be to add a binary output file option to
the operating software. For starts, the ASCII format would be the default-
those of us who needed/wanted the binary format could select it. If there
is no outcry, then ASCII would remain the default. Regardless, the ASCII
output would remain available.

The issue of the radial positions is one of compacting the data sets. Only
the absorbance system uses a sensor to determine the radial position.
Beckman is honest in putting the converted output from that sensor into the
output files as the radial reading. The interference and fluorescence
systems calculate the radial position- for a binary file it is far more
compact to save the parameters needed to reconstruct the radii than it is
to save each number. There would be no loss in precision. Though I could
make a strong case that the same is true for the absorbance system, I can
fully appreciate the desire to keep the real data. A different binary
format might be used for the absorbance system. Since the binary format is
there for 'internal' use, it matters little whether all file types are in
the same format.

The issue of intensities versus absorbance is probably the result of my
experience with questions concerning the quality of absorbance data. The
vast majority of the questions can be cleared up by looking at the
intensities (a publication on this is available over the net at
http://www.beckman.com/biorsrch/prodinfo/xla/a_1821a.htm). Likewise,
obtaining high quality absorbance data is made a lot easier if the
intensities are examined before setting off to acquire a set of scans. This
isn't made easy with the present operating system, and the conversion from
intensity to absorbance is clunky (largely due to the ASCII files, by the
way). You are entirely right that the vast majority of users want to get
good absorbance scans and could care less about the intensities (rightfully
so, too!). What I want to make sure is that they get good quality scans. I
am certain that the absorbance scan will remain the default, and that it
will fall on us working on this to devise a means for helping users out.

We are at the start of this venture and I hope you will be an active part
of it!
Best wishes,
Tom Laue


Tom Laue
Biochemistry and Molecular Biology
University of New Hampshire
Durham, NH 03824
Ph:  603-862-2459
FAX: 603-862-4013