INSTR UMENTATI0 N
Advisory Panel Jonathan W. Amy Glenn L. Booman R o b e r t L. Bowman
Jack W. Frazer G. Phillip Hicks Donald R. Johnson
H o w a r d V. M a l m s t a d t Marvin M a r g o s h e s William F. Ulrich
i
Image-Analyzing Microscopes Philip G. Stein National Bureau of Standards, Washington, D. C. 20234
Image-processing technology, through expanded data gathering and storage capabilities, now has the potential to solve difficult problems in metallurgy, polymer and fiber chemistry, and cell biology. Many studies involving the counting and sizing of particles or fibers can be automated through the use of image-processing techniques in metallurgy, polynill. mer nnd fiber chemistry, and cell biology are being t:ickled today using m y PROBLEMS
image processing techniques. A typical niensurement of the area of voidi in a mctallic specimen, for example, nould require making a photomicrogr:ipli> printing it, :ind measuring the resillring photo. Some laboratories use electromechanical scanners to make thesc measurements on the print, or dirwtlj. from the negative. Elementnry tests may be done with logic in the scanner. N o r e sophisticated problems are attacked by making :t digital tape recording of tlie print and processing it with a general-purpose computer. +Idifferent appro:ich t o this type of problem hns recently been embodied in several instruments, knowi as imageannlyzing micro,scopes. At the very least, they can eliminate the time-consliming and inaccurate pliotograpliic steps by scnnning tlie specimen directl!. in the image plane of the microscope. Further sopliisticntion could lead to improi-ement of the datn gathering process with on-line computer control of the scanning and other microscope f 1111c t ions. Each of these instriiments contains a scanning micropliotometer, wliich di].ides the image plane of the microscope into a number of resolution element$ and measures the luminous flus transniittcd tliroiigli (or rcflected from) each element. Datx are then processed by special or general purpose computers connectrd directly to the microscope. Somc instruments include variilblen-:%vc-lcligtliillumination and a motor-
driven specimen stage iyliicli allows positioning of more than one specimen field under the microscope objective. Ail of these systems share one basic consideration that colors their design : the amount of data is immense. I n the S P E C T R E I1 system at the Sational Institutes of Health, a single scan a t one rravelength produces 2 l 9 information bits. A single scan in this case, resolving a t or near the limits of the optical microscope, covers only a 50x 50-pm square on the specimen. A 10- x 10-mm specimen would require 40,000such scans at each wavelength of interest to acquire d l of the data.
For this reason, most of the instruments do not store the rLiwdata They may, in fact, have provisions for recording or transmitting the data to a computer, but in normal operation they rely on the specimen itself to act as a By repeated data storage medium reference to tlie actual image, very little storage of the basic densit! -gobition d'ita 1s required Gathering the Data
The heart of any image-analyzing microscope is the scanner. It partitions the image into a number of equal
"..
WNIROL LOGIC ER GENERATOR
+?-e!=I SPECIAL OPTICS& SCANNER
ANALOG VIDEO PROCESSING
ILIGHT YEASUREYENTS)
ITHRESHOLD CONTROL, E M COHTROU
A TO 0 WNVERSION
INFORNATION IHFORYATIOII
YICROSWPE
YECHAXI CALV WSlTlONED STACE CMTROL (OPTIONAL ON SUE IHSTRUYENTS)
------
Figure 1. Image-analyzing microscope
IYAC AIALYZIIIE COMPUTER (YAY BE A EENERAL PURPOSE DIGITAL COYPUTER)
ANSWER DlSPLAl OR P R m U T
ANALYTICAL CHEMISTRY, VOL. 42, NO. 13, NOVEMBER 1970
103A
Instrumentation
areas and measures the light through each. The size of the area measured is a function both of the microscope objective and the scanning aperture. An aperture of 10-pm diam at the image plane, for example, corresponds to a 0.25-pm diam spot on the specimen (with a 40X objective). To take full advantage of the resolution implied by this aperture-objective combination, one must use large aperture, oil-immersion objectives and condensers, and be extremely careful with the incident illumination. All image-analyzing microscopes require the operator to set up the condenser system manually, and the resultant measurements are somewhat dependent on how well this is done. For less critical problems, smaller magnifications and apertures obviate many of these problems. The user must also make some trade-off between resolution and contrast. At high resolution, the image contrast is reduced, and more amplification is required in the scanner light-measuring circuit. This, in turn, results in increased noise. iigain, lowerresolution studies are relatively easy. Types of Instruments
MC&B sends
Greetinas and if you want i
his beautiful miniature holly tree
Why deck the lab with boughs of holly when you can have this beautiful miniature holly tree. Just the thing to add a festive air to the area. To get yours send just one dollar and any MC&B symbol (from an ad or label). Don't delay and you'll get your tree in time for the holidays. Use the handy coupon.
CHEMISTS Norwood. Ohio. Baton Rouge, L a , East Rutherford, N J , Los Angeies, Calif
r"""""""-"'""
i f
I
5-70 MCB I
MC&B MANUFACTURING CHEMISTS 2909 Highland Avenue Norwood, Ohio 45212
Send m e my miniature Holly Tree
I I NAME I I COMPANY I ADDRESS
I
CITY
IL--STATE ---,,-,,,,,,,,,,,---,,J
ZIP
I I I I I I I I I I I I I I I I I
Figure 1 is a block diagram of an image-analyzing microscope. Clearly, the technical details of the equipment following the scanner depend on the scanner design. Breaking these into broad categories, they are T V scanners (raster), electronic scanners (using a random-scan tube), and mechanical scanners. The TV scanner is most amenable to use mit,h special purpose computers. The data are scanned in a raster fashion, completing one line of X for every increment, in Y until the entire field is covered. This is done a t a fixed rate. For vidicon-type camera tubes, this is important, as the tube photocathode integrates all of the light reaching each position. When a given spot is interrogated by the beam, the information is erased and a new integration period begins. For the dntn to be uniformly noisy (or quiet), the integration time a t each point must be the same. Therefore, use of a vidicon for random scans is not recommended. Since each line is being scanned at a fixed rate, an object in the field will be transformed into a time series of voltages representing the transmission at, each point. If we set a threshold (Figure 2) which says "anything darker than this level is an object," we then get a pulse whose width is determined by the object size, the scanning rate, and the threshold setting. This is the basic principle behind the TV-type special computers.
Circla Ma. 87 aa R m d m Stnice Crrd
104A
ANALYTICAL CHEMISTRY, VOL. 42, NO. 13, NOVEMBER 1970
1
[plenum PUBLISHING CORPORATION
DEVELOPMENTS IN APPLIED SPECTROSCOPY* VOLUME 8 Edited by E. L. Grove, Illinois Institute of Technology Research Laboratories, Chicago, 111. A new volume i n this important continuing series, Volume 8 presents both theoretical and applied principles of emission, atomic absorption, x-ray, nuclear particle, Raman and infrared, nuclear magnetic resonance, and electron spin resonance spectroscopy. I n addition, the book reports recent advances i n the areas of computer applications; instrumental applications to biomedicine toxicology; spectra characterizations; and air and water pollution. New findings on trace element analyses, silicate analyses, Mossbauer techniques, reference spectra, and retrieval systems are also included. Proce,edin s of the 20th Annual MidAmerica gymposia on Spectroscopy, held in Chicago, Illinois, May 12-15, 1970 $20.00 325 PAGES OCTOBER 1970 SBN 306-38308.X
ANALYTICAL CALORIMETRY *
VOLUME 2 Edited,by Roger S. Porter, Head, Polymer Science and Engineering Program, University of Massachusetts, Amherst, Massachusetts and Julian F. Johnson, Department of Chemistry, University of Connecticut, Storrs, Connecticut
Volume 2 brings research workers i n this dynamic area a broad, new perspective on current activities. The work covers a wide range of materials, including: studies of polyfins, polymer single crystals, copolymers, polyblends, elastomers, epoxy resins, and a variety of thermosetting polymers; special reports on low molecular weight organic compounds, organometalics, inorganic salts, and metal alloys; descriptions of related thermal techniques, such as thermal depolarization, microscopy, thermal mechanical and thermogravimetric techniques, as well as illustrations of t h e varieties of instrumentation used i n analytical calorimetry including high-pressure operation, automa. tion, and data interpretation; and the use of thermoanalytical equipment, including applications of combined differential scanning calorimetry and mass spectroscopy. Proceedings of the Symposium on Ana-
lytical Calorimetry a t the meeting of the American Chemical Society, held in Chicago. Illinois. September 13-18, 1970 460 PAGES SEPTEMBER 1970 $19.50 SBN 306.30366-3 Place you,r continuation order today for books in this series. It will ensure the delivery of new volumes i m mediately upon publication; you will be billed later This arrangement is solely for our convenience ,and may be cance{ed by you a t any time.
*
VISIT US AT BOOTH #9 AT THE EASTERN ANALYTICAL SYMPOSIUM
[plenum press/consultants bureau Divisrois
01
1
Plenum PbbI Gh r g Corpordt o n
227 W.17th ST., NEW YORK, NEW YORK 10011 Circle No. 117 on Readers' Service Card
Instrumentation
the data at each different' setting, thus generating "local" thresholds in the image. This is cumbersome, though. I t is better to approach those problems requiring this degree of sophistication with a different instrument. Other Ways to Do It
I
OBJECT
~
Figure 2. Object detection
I
1
1
I
1
I
1
~
VOLTAGE
TIME
SCANNED YlDEO SIGNAL
I I
TIME
Analysis-TV Microscopes
From this point, there are many ingenious n:iye t o process the pulse data using logic rirniits. Some of the possibilitics are : ( 1 ) 17ying a TV display of the image, intcnsify or otherwise identify all points :ibove the threshold enahling the operator easily to find a proper tlircshold setting. (2) Similnrly intensify only tlie leading or trniling edge of an object. (3) Restrict the imnge processing to a rectnngiilnr rubportion of the screen, and indicate the area to the operator. The operator may continiioiisl!. adjust this "nindon-." ( 3 ) Reject pulses shorter than a cert:iin minimum kngtli, thereby ignoring much dirt and some artifacts of the specimen as being too small. ( 5 ) Save the previous scan line in a memory and compare it with the current one. If an object \vas present on the lnst line and is still present, do not count it ag:iin. With t h r s c arid other processing techniques, it is possible to automate many problems involving counting and sizing of particles or fibers. By rotating the image 90G, either mechanically or optically, measures of out-of-roundness or form factor are also easily generated. The data may be transferred by an
automatic sequencing programmer to .I desktop calculator, statistics de\-eloped about the field, and measurements wliicli are numerically derived from two or more pulse variables can be calcul,ited I n addition, the data may be transferred to a general purpose computer through ,in approprlate mtcrface, and further computations donc~ Why Not?
The most serious problem rvith instruments of tliis type lies in their assumption of data uniformity. The threshold control must be set manually by an operator, and so must the illumination. This is indeed true of any image-gathering device of this type, but tlie TV system relies very heavily on this setting for the repeatability of the data. I n addition, the logic must depend on an implicit requirement of spatial uniformit!. of both specimen and illumination. The threshold remains uniform over the field, even though the data may not be. Another problem is that, even with the most sopliisticnted logic now available, it is not possihle to recognize donuts or otlier miiltii,ly-connected shapes as just one object. TTliether this is of any conseqiiencc, of cour:'c, depends o n the prohlein. It would hc possible to connect a computer in siicli a way that it could adju5t tlie thresliold and interpret
Iiandom-access scanners may use a moving specimen stage, vibrating mirrors in the light path from objective to image plane, or an image-dissector scanning tiibe. Of these, only the im:ige dissector shares with the vidicon the problem of photocathode nonuniformity, hut, it has a great speed advantage over the mechanical scan methods. In nll cases though, the oiitput of the scan is trnnmitted directly to a general-purpose digital computer. I n some instruments, this computer can directly control the scan pattern. It c n n also be connected to make large motions of the specimen stage (surh as from field to field), change the color, intcnLity, aiicl bandwidth of the illumination, and change the focus. S o n e of these systems has a storage photocathode, so there is no need to scan every point in the field at n fixed rate. The transmitted or reflected intensity darn are gathered directly by the compiiter, and the image processing may he done either tlireetly or by transmitting tlie data t o a large, high-speed machine. This approncli is far mnre complie:itrtl :ind rspcnzive than the purchase n n d operation of a simple TT' micros r o p . I t i,? h o much more flesible, permits a great deal of investigation into the interaction of the machine and the properties of the data, and allows unliniitcd sophistication of the imageprocessing tecliniqiies employed. These might involve, even on the most basic level, continuously variable local thresholds, depending on the data t,hemsclvcs, thrcsholds of sharpness (rate of change of light) rather than simple intensity, nnd programs that follow boiindaries with the scan to resolve ambiguities about shapes such as the donut. Since the data nre available in thcir rnw form, the computer might select a proper n-avelength for n stndy on the basis of contrast, or it might adjust the integration time at each point in the image t o improve .tatistics while w s t ing little or n o time on background area.?. The computer also might instruct the scnnner to look at a field with very low resolution (perhaps every tenth line), and locate all of the large objects. The scan would then return to those areas where data were found to scan them in detail.
ANALYTICAL CHEMISTRY, VOL. 42, NO. 13, NOVEMBER 1970
105A
There can be no water pollution control
..,until you first know exactfy wh The TechniconYXM”6 is the pe
Hardness.
m
principle perfected in the Technicon AutoAnalyzer,” basic models of which are available for analysis of hand collected samples. For further information, please write to Depa
Technicon Industria Division of Technicm
Tarrytown, New York 53 Circle No. 141 on
R
W Ser
Instrumentation Artificial Intelligence
It is also possible, in some special cases, to use the information about geometry and spectral transmission or reflection to identify and classify isolated specimens. This hopefully would lead t o the ability to distinguish normal and abnormal specimens automatically. This development depends strongly on processing the image in a programmable fashion, making it easy to change classification techniques. A very general facility as in Stein et al. ( 1 ) could be used to simulate the operation of any number of specialized image-gathering and image-analyzing microscopes. I t could be used, therefore, as a design tool for single-purpose instruments. Finally, the computer can produce a display for the operator which shows not only the raw data plus a threshold, but also the result of the data processing by any algorithm available to the programmer. I n this way, the imageanalyzing techniques themselves may be iteratively improved.
If a metering pump is part of your system, make sure the pump matches the high performance standards of your other instruments. When you choose an advance design Buchler pump with uniformly adjustable bow rates, you have *at assurance. Write fbr complete information.
POLYSTALTIC' PUMP) lPumps
liquids and gases in many applications including column chromatography and automatic analyzers
Peri~%ltoc Aaion through 4 veins
Pbw capadty from 2.5 to Literature Cited
lo00 ml/hr/vein
(1) P. G. Stein, L. E. Lipkin, H. M.
Shapiro, Science 166, 328-33 (1969).
~BEKASTALTIC~ PUMP Suggested Source Material
lUsed
in column chromatography, aminwcid analyzers, IiquM transfer to 10 different locationr and infusion of animals
(1) R. C. Bostrom, H. S. Sawyer, W. E.
Tolles, Proc. ZEEE 47, 1895 (1959) ; P. H. Neurath, B. L. Bablouzian, T. H. Warms, R. Serbagi, A. Falek, Ann. N . Y.Acad. Sci. 128, 1013 (1966). (2) M. L. Mendelsohn, B. H. Mayall, J. M . S. Prewitt, R. C. Bostrom, W. G. Holcomb, Advan. O p t . Electron Microsc. 20, 77 (1968). (3) M. Ingram, P. E. Norgren, K. Preston, Jr., Ann. N . Y. Acad. Sci. 157, 275 (1969). (4) G. L. Wied, P. H. Bartels, G. F. Bahr, D. G. Oldfield, Acta Cytol. 12, 180 (1968). (5) W. Prensky, J . Cell Biol. 39, 157a (1969). (6) K. Preston, Jr., P. E. Norgren, Ann. iV. Y. Acad. Scz. 157, 393 (1969). ( 7 ) L. E. Lipkin, W. C. Watt, R. A. Kirsch, Ann. N. Y. Acad. Sca. 128, 984 (1966). (8) L. E. Mawdesley-Thomas, P. Healey, Science 163, 1200 (1969). (9) S. A. Rosenberg, K . S. Ledeen, T. Kline, Sczence 163, 1065 (1969). A publication of the U. S. National Bureau of Standards. S o t subject to copyright. Discussion or description of specific instruments herein should not be construed to indicate that these instruments have been tested or otherwise evaluated by the National Bureau of Standards, and no endorsement or criticism of any specific manufacturer is intended or implied.
Peristaltic Action through 10 vsins plow capaclty from 1.5 to 650 m I /hr/ve'n
MICROPUMP) &ten organic solvents through chromatographic columns a t preset rates Liquid in contact with glass, Teflon* or Kel-F** only
Pumps against 20 psi. Flow rate of each chamber can be set independently Tied8 Merk E. 1. DuPont De Namours. Inc. Minnesota Mining 6 Mfg. Co.
** Trade Mark
BUCHLER INSTRUMENTS DIVISION
0 BUCHLER INSTRUMENTS
N U C L E A R - C H I C A ~ OCORP. A S-BStOARV
OF G 0
ESEAR-E IIIl
6 CO
1327 SIXTEENTH STREET, FORT LEE, NEW JERSEY, 07024 ~
~~
Circle No. 37 on Readers' Service Card
ANALYTICAL CHEMISTRY, VOL. 42, NO. 13, NOVEMBER 1970
107A
