From: Philo, John <jphilo@amgen.com>
  To  : 'rasmb' <rasmb@bbri.harvard.edu>
  Date: Wed, 9 Apr 1997 10:37:11 -0700

binary formats: how slow is ASCII?

I will try to summarize some discussion between Walt Stafford and myself
yesterday that I think may clarify to what extent the ASCII format
presents problems with slow data loading, particularly for those who may
use alternate software or computing platforms.  

The slower data loading with ASCII files becomes particularly
problematic when one needs to simultaneously analyze ~100 interference
data sets as is often done with Walt's DCDT software.  Walt did some
testing yesterday and tells me that it takes his Macintosh version of
DCDT ~3 minutes to read in 100 interference scans in ASCII format, but
only 4.5 seconds to do the same scans in binary format.  For Mac users,
the situation is also aggravated by the time it takes to transfer the
large ASCII files from the PC.

However, it is also important to re-emphasize a point already made by Bo
Demeler: the rate at which one can load the existing ASCII format is
strongly dependent on the hardware and software.  I tested the ORIGIN
version of DCDT on the 133 MHz Pentium supplied by Beckman and (by
extrapolation) it would take about 7 minutes to load 100 files. Jeff
Lary's DOS version of DCDT loads data at a rate that corresponds to < 10
seconds for 100 scans, and my XLGraph interpreted BASIC routine can load
100 files in about 35 seconds.  Therefore there are differences of at
least 70-fold, even on the same hardware.

Thus whether or not this is a "problem" depends on numerous factors:
what type and how many experiments one is doing, what software/hardware
platform one is using for the analysis, and ultimately personal
perceptions of what is "too slow" to be tolerable.

John Philo