From: UMJ <xuyu@UMDNJ.EDU>
  To  : Peter Schuck <pschuck@helix.nih.gov>
  Date: Fri, 28 Apr 2000 15:42:54 -0400

Re: linear approximation for error estimates

> With regard to Yujita's comment about the validity of the "linear
> approximation", I'm very much surprised that this should be the case.
> Maybe there is a misunderstanding.  I can see that with large data sets
and
> with Gaussian noise the statistical assumptions underlying least-squares
> optimization is valid, but not that the actual confidence limits are those
> from "linear" least-squares.
>
> The reason is that by simply mapping the error surface for an equilibrium
> model, i.e. if you keep one parameter fixed at a non-optimal value,
> optimize the others and observe the chi-square of the constrained fit, I
> find that a symmetrical parabolical minimum is really the exception (the
> linear least squares implies a symmetric parabolical minimum).  I think
> Michael Johnson has worked a lot on this, and in his book chapter
"Comments
> on the Analysis of Sedimentation Equilibrium Experiments", he explicitely
> says that "[the asymptotic standard errors from the covariance matrix]
> almost always significantly underestimate the true confidence intervals of
> the determined parameters" (Todd Schuster and Tom Laue's book, Birkaeuser,
> 1994, p. 51)
>
Yes, it seems there is some misunderstanding. I am glad you agree with me on
that 'with large data sets and with Gaussian noise the statistical
assumptions underlying least-squares optimization is valid'. This is all
what I was trying to say in terms of asymptotic linear approximation.
You are also right (so is Michael Johnson) that 'the asymptotic standard
errors from the covariance matrix almost always significantly underestimate
the true confidence intervals of  the determined parameters'. However, I
think the problem here is the term 'confidence interval'; covariance matrix
is a theoretically well defined concept, while 'confidence interval' is not.
When we say the covariance matrix of a nonlinear least square fitting
program is an asymptotic approximation of linear least square fitting
procedure, it does not mean you can just take out the diagonal elements of
the matrix and equal them to some kind of 'confidence intervals'. I think
the trouble with confidence interval has more to do with the multi-dimension
(n>2) of the matrix than with the non-linear estimation. For a linear
regression you will see the same thing: the covariance matrix significantly
underestimate the confidence intervals based on the definition of
'confidence interval' by Mike Johnson (I know so, since I did such tests
with NONLIN).

Regards
Yujia