Itatistical Desinn Gun problem illustrates the importance of proper scheduling of measurements by W. J. Youden XPOSITIONS
on experimental design
E often present the basic principles of design on the assumption that the reader will have no difficulty in imagining experimental situations in which these principles may be successfully applied. This month's column describes an experimental situation and considers various ways of conducting the investigation. The purpose is to show that disregard of the principles of statistical design may lead to inefficient experimentation and misleading results. Most experimental situations involve uncontrolled factors
The gun problem refers to a test program to determine the relative merits of five new ammunitions. It is postulated that the ammunitions are for use in large 16-inch guns. The quantity of ammunition required per round is such that only four rounds are available for each ammunition. The large gun is specified as the test weapon because the gun barrel wear is fairly pronounced and cannot be ignored in the work. Indeed, in order to simplify matters, it is assumed that the wear has a linear effect on the observed performance which might be the velocity of the shell. It is assumed that barrel wear is the only consequential uncontrolled environmental factor. Imagine the experimental results that might be obtained if 20 rounds of the same ammunition are fired in succession in this one gun barrel. Apart from minor fluctuations in the data, there should be evident a steady falling off in the recorded velocity. The observed velocities, u , when plotted against the firing order should lie closely along a down sloping line. Suppose now that the 20 rounds of the same ammunition are replaced by four rounds of ammunition A, followed by four of B, and so on until the work ends with the firing of four rounds of ammunition E. The firing order of the letters is shon-n along the x-axis. August 1955
It should be immediately clear that the several ammunitions are now identified with and are the recipient of certain effects that arise solely from the firing order employed. The situation is really worse than it appears. The bunching together of the four rounds of any one ammunition tends to give four velocities in fairly good agreement. Xumerical evaluation of the data may lead to the acceptance of differences between the averages of the five ammunitions when these differences are judged by the close agreement shown by repeat rounds of the same ammunition. The ammunitions might be identical, the observed diff erences arising solely fr0.m the firing order. Nothing good comes from this work. The averages are worthless. Each average depends on its position in the firing order. The estimate of the experimental error based on repeat rounds fitted in succession obviously has no proper applicability for judging differences between ammunitions not fired in immediate succession. It may be maintained that the above program constitutes such a flagrant violation of good experimental practice that no one with any experience would blunder so badly. Yet horn often workers insist on some systematic schedule simply because this will reduce the chance of mistakes in recording and in general will simplify the conduct of the work. So very often duplicate analyses are run on one material today, on another material tomorrow, and so on through the week.
The trend through the week may not be linear. The use of the error of the duplicates to judge the several averages assumes that there is no trend. The points raised above are the basis for the emphatic rejection of the firing schedule listed above. There are altogether 305,540,235,000 distinguishable firing schedules, including 119 more schedules in which like rounds appear together in sets of four. Is it enough to set aside these few systematic sequences and pick a t random a firing schedule? For example, 20 marked pieces of paper drawn from a hat gave D E B A B C A E A D CC D B A EBCED Random element i n the order serves t w o purposes
There are two important consequences of the randomizing procedure. First, a random order does give all rounds fired an equal chance at each of the 20 positions in the firing order. If the effect of gun barrel wear is not linear but some unknown complex curve, then a random order, in essence, samples the available positions fairly for each of the ammunitions tested. The second consequence of randomization is the effect on the estimate of the experimental error. A random order scatters like and unlike rounds in the sequence, so that the separation between like rounds tends to be the same as the separation of unlike rounds. Possible differences between unlike rounds must be judged against
1 2 3 FIRING ORDER 20 A A A A B B B B C C C C D D D D E E E E
INDUSTRIAL AND ENGINEERING CHEMISTRY
103A
Statistical Design the differences observed between like rounds. Consequently, if the different ammunitions should, in fact, be identical in performance, the experimenter now has the best protection against reporting differences that do not exist among the ammunitions. The point will be made by many experimenters that something better than a purely random order can be devised if the results are subject to a linear fall off arising from barrel wear. Any arrangement of the letters in the first ten positions that is followed by the letters in reverse order will compensate exactly for the disturbance introduced by the linear trend. Thus, ABCDEGEABDDBAECE DCBA
i
TO
Solve i Problems i of ...
i
M A S S TRANSFER
involving
direct contact heat transfer
and absorption, as well as gas cleaning
by impingement
...
check Peabody Scrubbers first - Peabody engineers, with years of experience in industries throughout the world, have consistently solved hundreds of complex gas processing problems. Service and maintenance are practically eliminated since Peabody uses no moving parts. Low liquid pressure and minimum consumption result in maximum economy. Every job is custom-engineered!
Here's why
Here's how
- gas and
liquid meet with extreme turbulence at Peabody's impingement baffle plates. These provide as many as 500,000 scrubbing jets for gas-liquid contact and insure maximum cleaning, cooling and absorption.
PEABODY ENGINEERING CORPORATION 580 FIFTH AVENUE, NEW YORK 36, N . Y . OFFICES I N PRINCIPAL c i r m PEABODY LIMITED LONDON, S.W. 1 , ENGLAND For further information, circle number 104 A on Readers' Service Card, page 115 A
104 A
ensures that the average ordinate to the sloping line is the same for each letter. The repeat runs of a given letter tend to be more widely separated. The variation contributed by gun barrel wear no longer affects the letter averages. This variation has all been forced into the disagreement between results of like rounds. Clearly if this arrangement is to be useful some numerical procedure must be devised for evaluating the experimental error without including the component associated with gun barrel wear. There is a simple way to disclose the experimental errors other than that contributed by wear. Form the ten differences by subtracting the 20th from the 1st result the 19th from the 2nd result
.....................
.....
the 11th from the 10th result
Each subtraction involves two rounds of the same ammunition; therefore the velocity of that ammunition drops out. The first difference involves the extreme positions in the order and should reveal the fall off in velocity produced by 19 rounds. The second difference shows the fall off in velocity produced by 17 rounds. The last difference gives the drop in velocity produced by one round. If these ten differences are plotted against 1, 3, 5, . . ., 19 on the z-axis, the linear trend from wear should be evident, obscured only by deviations ascribable to other sources. That is, a straight line may be fitted to these 10 differences and the deviations of the plotted points from the straight line provide a means of estimating the experimental error appropriate to the
INDUSTRIAL AND ENGINEERING CHEMISTRY
Vol. 47, No. 8
comparison of the ammunition averages. If wear produced a truly linear effect and there were no other sources of error in the measurements, these dil’ferences would lie exactly on a line. There is an interesting example of the use of this reversal of the order of the measurements. Laboratories calibrating precision thermometers prefer to take the readings on a rising meniscus. It is, therefore, customary to place the thermometers to be calibrated along with a reference standard in a bath that is very slowly rising in temperature. The thermometers are read in any convenient order and immediately reread in reverse order. Provided that the rate of rise is constant and that the interval between readings is kept constant, the averages for the two readings for each thermometer are as exactly comparable as they would be if the bath temperature had been kept constant. The average temperature for each thermometer is that of the middle of the interval just preceding the repeat measurements in reverse order. Under these circumstances this reverse design is as effective a sequence as can be devised. Where does the random element appear? There are a number (I 13,400) of sequences that satisfy the requirementand oneof these is picked at random. Of course, knowledge of the conditions made it possible to restrict the choice of sequences to a particular small class. More often there is little or no information regarding the nature of the trend. The systematic order (AAAABBBBCCCCDDDDEEEE) and the symmetrical reverse sequence (ABCDECADBEEBDACEDCBA) represent two extreme situations regarding the effect of gun barrel wear on the measurements. With the systematic order, gun barrel wear has the largest possible effect on the averages and a minimum effect on the agreement of duplicate rounds. With the reverse sequence, gun barrel wear does not disturb the averages but does introduce maximum disagreement between duplicate rounds. A random order avoids both these extremes and tends to apportion the effect of gun barrel wear equally on averages and the agreement of duplicates. This is true even when the wear effect is nonlinear. Correspondence concerning this column will be forwarded if addressed to the author, % Editor, INDUSTRIAL AND ENGINEERING CHEMIBTRY, 1155-16th St., N.W., Washington 6, D. C.
August 1955
when there’s something
/ /
use
i i
\\
C 0 1UMB IA activated carbon Trade-Mark
Yes, when thrre are organic lapors in an atmosphere, and you want to remove them-think first of’ COLUMBIA Activated Carbon with its extremely high adsorbing capacity. Solvent vapors from your manufacturing operations can be recovered for re-use at great savings. Odors, impurities, and toxic vapors can be removed from air and industrial gases; h j drocarbons separated from natural gas, and refinery or chemical-plant feed streams-economically and efficiently. And, as a catalyst carrier, C O L C M RActivated I~ Carbon is outstandingone cubic inch of a typical pelleted form has a calculated active surface area of more than 88,000 square feet!
COLUMBIA Activated Carbon is hard, durable, inert. It has low ash content and is not affected by most chemical agents and conditions. It is supplied in a wide range of porosities and in particle sizes ranging from a fine powder to 3/16-inch pellets. Activated Carbon. Return the coupon today. Get the full story on COLUMBIA
C O LUMB IA activated carbon Trade-Mark
is u product of: CARBIDE AND CARBON CHEMICALS COMPANY A Division of Union Carbide and Carbon Corporation 30 East 42nd Street lI@l New York 17, N. Y.
AN0
CARBON
W
Please send m e the booklet “COLUMBIA Activated Carbon,” Form 471 2.
Company
Position
Address
“Columbia” i s a registered trade-mark of Union Carbide and Carbon Corporat!on. For further information, circle number 105 A on Readers’ Service Card, page 115 A
INDUSTRIAL AND ENGINEERING CHEMISTRY
105 A
