From: Tom Laue <tom.laue@unh.edu>
  To  : rasmb@bbri.harvard.edu
  Date: Tue, 08 Apr 1997 09:02:22 -0400

Binary format

Dear Rasmbers,
John's request is very timely. I am in the process of writing the operating
software for the fluorescence detector on the XLI. It is my hope to rewrite
most of the user interface to take advantage of 32-bit math and Win95
multithreading to allow the different optical systems to each run at its
own pace. During this I have come to the conclusion that there are a couple
of problems looming for data collection and analysis. The use of a standard
binary format (with appropriate conversion software for backwards
compatibility) will minimize some, but not all, of these problems. A few
thoughts are included here as a start for the discussion. Many of the ideas
are from Walt Stafford, Jeff Lary and John Philo.

A second concern is the structure of the directory tree holding the data
files. I have done some thinking about this problem and have come to the
conclusion that it might be best to dispense with a directory tree. My
thinking is as follows: a) there are at least two equally valid tree
structures for organizing the data files (one in which the second level
corresponds to the cells and one in which the second level corresponds to
the optical systems and b) even with the directory tree as an organizing
structure, there are many times when data analysis requires combining files
from different branches, making it awkward to choose or use either tree
structure. Part II describes my proposal for overcoming these obstacles.

Finally, Preston Hensley and Jim Lee have proposed that a round table be
convened at the Galveston meeting to discuss these matters. I hope that
this round table can be the place where these ideas are cemented, and that
the RASMB discusion will be the format for hashing out proposals. We need
to get this put in final form quickly- there is a real danger of discussing
this past reason.

I. File format proposals
	A. the binary file should contain enough information to reconstruct a
backwards-compatible ASCII file
		i. Implication: the present headers should be stored with the file
	B. the radial information should be encoded as a start, increment and
number of points 
		i. A concern: this violates the 'true' values returned by the absorbance
encoder
		ii. Wavelength scans would be stored in a separate format
	C. real numbers are stored as IEEE binary values (double or long double?)
and restored to floating during conversion
		i. long double (64 bit) is not a standard for all compilers and OSs
	D. the third column of interference data should be the Fourier magnitude
	E. absorbance data should be stored as intensities
II. File database proposal
	A. the data acquisition system should create a database (in a standard
format- ODBC, Access) of file names
		i. the files from each experiment will then be kept in their own directory
	B. the database should also contain all pertinent information about each file
		i. optical system
		ii. time
		iii. cell
		iv. channel
		v. comment
		vi. rotor speed
		vii. temperature
		viii. wavelength 
			a. Excitation and emission for fluorescence
		ix. w^2t
		x. others??
	C. the information will serve as keys for sorting the database
		i. keys will be lockable, allowing sequential sorting by keys (e.g. rotor
speed/cell/optical system)
	D. file conversion and data analysis programs will be launchable from
within the database
		i. selected files will be used as input to the program
		ii. files will be processed sequentially (e.g. an editor) or
simultaneously (e.g. Nonlin)
	E. Either files or a list of file names may be exported as an ASCII file

Have at it!
Best wishes,
Tom Laue

Tom Laue
Biochemistry and Molecular Biology
University of New Hampshire
Durham, NH 03824
Ph:  603-862-2459
FAX: 603-862-4013