CORRESPONDENCE SIR: Opfell and Sage [IND.ENG.CHEM. 50, 803 (1958)l use a rather different and unusual least squares nomenclature. I would like to point out a few errors. O n page 804, Z(AyJAyJ2 should be Z(Ayi/
Applications of least Squares Methods Applying the above theorems to Equation 2 with a and b as the variables and constant x , vu2 =
Uo2
=
go2
uAY~)’. The authors deal with the treatment of data of varying precision. This is needed when one has both high and low accuracy test data or uses averaged data which have different numbers of points in each average. They rightly suggest the use of weighting factors to emphasize the more precise data. Their Equation 22 shows how to extend ordinary least squares to weighted least squares. In ordinary (nonweighted) least squares, a measure of agreement between prediction and experiment is the variance of estimate (see Equation 14 below). This is reasonable (Equations 3, 11, 12, and 13) because it is directly related to the confidence interval for prediction. The authors have tried to extend this measure by using a “weighted average variance of estimate.‘’ The weighted estimate is not related to confidence limits and may not be used to measure how well you can predict (case V). The prime question is, “HOWgood is your prediction?” Statistically this is answered by using confidence limits. Suppose y is truly dependent upon x :
+ Bxi + ei
yi = A
(1)
where e; is a random variable (experimental error) with mean = 0 and variance = ui2. We are given a number of data points (vi, xi) and wish to estimate A and B. If cr,? is constant over all i, we choose a and b (estimates of A and B ) such that Z ( y i - a - bxi)z is a minimum. If ui2 is not constant over all i, we put more weight in the points with lower ui2 by minimizing Z(yi - Q - b ~ i ) ~ / uDe~~. pending on the nature of ui2,we choose one of these methods. What will be a measure of the fit of the data to Equation l? The authors correctly state that it is uu2,the variance of y as predicted from a bxi. Variance is defined as
+
=
u*2
E [ t - E(2)]*
where E is expectation. Two theorems may be derived from this definition. (1) For two random variables, m and t , variance of m -j- z is u,”,,
= u.2
+
am2
+ 2um
where uZm = covariance of z and m = E l m - E(m)][t- E ( z ) ] = 0 if m and t are independent. ( 2 ) For a random variable, t, and constant, k, the variance of kz is U E , ~=
k’u.2
y will be given by yi
226
=
(I
+
b ~ i
(2)
+ +
Uzb2 x2Ub2
+
+
200.62
(3)
2XUab
from the definition of variance and covariance ao2 = E [ a - E(.)]’ E(. - A)’ (4) E[b
Ub2
ua5 =
-
E(b)I2 = E ( h - B)’
(5)
E [ Q - E(Q)] [b - E ( b ) ] = E(. - A ) ( b - B ) ( 6 )
However, these equations do not allow computation of uu?. Case 1. ui2 = a constant over all i = u:. We minimize S = Z(yi - a bxi)* by taking derivatives, setting to zero, and solving for Q and b.
as/&
= Zyi
&Slab = Zyyixi
- Na - bZxi = 0 - Baxi - hZxi2 =
0
or in matrix notation
estimates made from the predicting equation. Using a Student’s t table we can give confidence intervals based on uy2. Let us write the equations for u2, etc. We minimize s = Z ( p - a - bXi)ZWi wherewi = 1/ui2 bS/Ba = Z w i j i - aZwi - Zwixi = 0 b S / a b = Zwixiyi - aZwiji - ZWiXi2 = 0 or
( Zwi?i,) (z Zwi wixi zwty,x,
=
or Y = X X C and c = x-‘Y where X-‘ = 1 ZWiZWiXi2 - (ZWiXi)2
Y = X X C
c
and where
x-1 =
=
l/[NZx,2 -
X-IY
(ZXi)Z]
ZJJy,ZXi2- ZX,ZYiXi
a =
N
b =
ZXiZ
-
(2Xi)Q
N z p x i - ZXiZJJi N ZXi2 - ( 2 X i ) Z
(9) (10)
Applying Equations 4, 5, and 6 to 9 and 10, by replacing y i with e i and taking the indicated expectations, since E(eJ2 = u:, E(ei) = 0, and E(e;ej) = 0, we get u2 =
dllU*2
(11)
a f = dzzu,2
(12)
uab = d12a.2
(13)
These results may easily be generalized to the multivariate problem. UZX-’, because of its nature shown by Equations 11, 12, and 13, is called the variancecovariance matrix. A measure of u62will be provided by S :
Z(yi - u - bxi)2/N - 2
(14)
Using Equations 11, 12, 13, and 14 we can now evaluate uv2,at any x , using Equation 3.
Case 2. ui*varying with i. Equation 14 gives a measure of how well the data fit the predicting equation. However, the modification of 14 expressed in the authors’ Equation 27 is not desirable. If gi2 is not a constant, but = uiz for any i, why get a weighted average variance? Instead derive cryzfrom Equation 3. This allows one to give confidence intervals on
INDUSTRIAL AND ENGINEERING CHEMISTRY
(15)
dl,
=
dip
(17) (18)
d??
(19)
There is no u: in Equations 17. 18, and 19. Furthermore, no ucL is needed. Use of these equations and 3 permits one to make suitable confidence statements.
989 Overlook Rd. Berkeley 8, Calif.
From Equation 7
(‘)
ua* = uob =
(7)
)
a and b are given by Equation 15. Applying Equations 4, 5,and 6 with the knowledge that E(ei)? = ~ $ 2= l / w % , we obtain u5’
or
Suixi BU’,Xt2
ALFRED M. TURKEL
SIR: Most of Mr. Turkel‘s suggestions apply to normal regression analysis. They reflect his considerable experience in this field. In most cases which we discussed. the magnitudes of approximation errors could be determined rather precisely. Thus our application of least squares methods differed from normal regression analysis in one essential-viz., the approximation errors were not random. It is this property of the systems which prevented us from applying the analysis suggested by Mr. Turkel to obtain “a reasonable over-all measure of agreement between prediction and experiment.” The measures suggested in our Equations 27 and 28 are crude and useful only in determining the effects of changes in values of coefficients on the fit of a particular empirical equation. The method of least squares imposes no restraint on the maximum approximation error. If the fit is excellent in all but a small region, the error in this region might be huge without being reflected in the measures of agreement suggested. The method of least squares is a convenience. A method which restrained the magnitude of the maximum approximation error might be of greater value for the ultimate purposes of prediction. Cutter Laboratories JOHNB. OPFELL Berkeley 10, Calif.
