Ind. Eng. Chem. Res. 2003, 42, 5345-5353
5345
Softwood Lumber Grading through On-line Multivariate Image Analysis Techniques M. H. Bharati and J. F. MacGregor* Department of Chemical Engineering, McMaster University, Hamilton, Ontario L8S 4L7, Canada
W. Tropper Tembec Inc., Temiscaming, Quebec J0Z 3R0, Canada
The concepts behind the use of multivariate image analysis (MIA) techniques for on-line monitoring of subtle features in time varying images were presented in an earlier paper (Bharati, M. H.; Macgregor, J. F. Ind. Eng. Chem. Res. 1998, 37, 4715). This paper illustrates the successful application of these ideas to an industrial process from the forest products industry. MIA is used to rapidly detect the presence and quantity of common lumber defects such as knots, splits, wane, pitch, and bark pockets in individual sawn lumber boards as they pass on a moving conveyor belt under a line-scan RGB camera. Multiway principal component analysis is used to decompose the acquired three-channel lumber images into a two-dimensional principal component (PC) space. Masking score point clusters in the score space and highlighting corresponding pixels in the image space of the two dominant PCs enables isolation of lumber defect pixels based on contrast and color information. Control charts with appropriate tolerance limits are set up to monitor counts of defective pixels lying under any defect mask in the MIA score plots for each new lumber sample as it passes through the imaging system. Those lumber samples violating the tolerance limits are automatically downgraded. This application of on-line MIA for assessing specific quality problems is not limited to lumber grading, but could be directly applied to many industrial processes involving the production of solid or heterogeneous liquid or liquid/solid products. 1. Introduction paper,1
In an earlier multivariate image analysis (MIA) methods were presented for on-line process monitoring. In that paper, landscape features such as bodies of water, roads, golf courses, etc., were identified and monitored from a sequence of multispectral images acquired by a moving LANDSAT (MSS) satellite over a region of the earth’s surface. The main focus of that paper was on illustrating the potential of using MIA techniques to visually monitor industrial processes that manufacture solid (or heterogeneous) products with the aim of improving product quality control. This paper presents an industrial application of those ideas to image-based automatic quality grading of sawn softwood lumber boards from the forest products industry. Wood has historically been one of the most popular building materials because of its various desirable properties, adaptability to a wide variety of uses, and relatively low cost. A typical sawmill produces lumber boards with varying degrees of quality, depending on the severity and distribution of defects. Correct grading of softwood lumber according to overall quality is of paramount importance because the difference in pricing between lumber grades is substantial. The quality of a lumber board is defined2 as a function of the highest grade and is then reduced by the occurrence of quality molding features (defects). The degradation in quality is amplified by the frequency, size, and * To whom correspondence should be addressed. Tel.: (905) 525-9140 ext. 24951. Fax: (905) 521-1350. E-mail: macgreg@ mcmaster.ca.
location of such undesirable features. Typical defects found in softwood lumber can be divided into three groups,3 namely (i) natural defects, which are caused by nature and develop within the living tree (e.g., various types of knots, pitch pockets, decay, wane, bark pockets, etc.); (ii) manufacturing defects, which are caused by equipment during the sawing and handling of lumber (e.g., raised grain, torn grain, fiber pull, machine burn, etc.); and (iii) seasoning defects, which occur when sawn lumber dries (e.g., splits, warped boards, etc.). The work presented in this paper addresses selected softwood lumber defects from two of the above three groups. They are knots, splits, wane, pitch pockets, and bark pockets. A “knot” as it appears on a piece of lumber is a portion of a branch through which the saw cut. “Pitch” is defined as an accumulation of resinous material in the lumber. “Pitch pockets” and “bark pockets” are well-defined openings between the annular growth rings of the tree containing liquid or granulated pitch or pieces of bark, respectively. A “wane” is defined as bark or lack of wood at the edges of a sawn piece of lumber. “Splits” are cracks that occur in the lengthwise direction of sawn lumber due to rapid evaporation of moisture from the wood surface. A complete list of the definitions and descriptions of most defects can be gathered from the NLGA Standard Grading Rules for Canadian Lumber.4 To obtain a distinct wood quality, the lumber board has to be introduced to a rule-based system, which accounts for the number, size, and position of its defects. However, because of the inherent variability in defects, as well as in the sound wood structure itself (both
10.1021/ie0210560 CCC: $25.00 © 2003 American Chemical Society Published on Web 09/18/2003
5346 Ind. Eng. Chem. Res., Vol. 42, No. 21, 2003
Figure 1. Schematic of a lumber vision system used to image moving lumber boards.
between and within species), it is difficult to automate such a system. As a result, lumber classification based on assigned quality grades has traditionally been a result of human judgment after visual inspection of each piece. Human lumber graders are highly trained individuals with many years of grading experience at making split second decisions about the quality of every passing piece of lumber at production speeds of up to 30 boards per minute.5 However, human errors and inconsistencies in grading lumber are common. Consequently, efforts have been made in the past decade to replace the human graders with imaging sensors and image analysis algorithms for automatic lumber grading. An excellent literature review of many proposed strategies and algorithms for automatic lumber grading via detection and classification of lumber defects has been provided by A° strand.5 Recently, Hagman2,6,7 recognized that lumber grading is a multivariate problem. He proposed MIA and multivariate image regression (MIR) techniques for extracting features from off-line multispectral images of lumber in the ultraviolet (UV) and visible (VIS) light wavelengths. The primary objective of this paper is to present MIA techniques for on-line monitoring of specific lumber defects using RGB color images of softwood lumber boards. The techniques provide both qualitative and quantitative results that can be used for automatically assigning quality grades to lumber boards at production speeds based on prechosen defective features. The approach consists of developing a robust principal component analysis (PCA) model that incorporates all of the inherent lumber variations. The model is used on-line to detect and isolate pixels corresponding to various defects on lumber boards imaged by a line-scan RGB digital camera. The proposed strategy is illustrated through the grading of 38 lumber boards from three species (balsam fir, white spruce, and jack pine) based on preselected defects including knots, splits, wane, pitch, and bark pockets. The paper is structured as follows. Some imaging system details are presented in section 2. Section 3 reviews the main concepts of off-line MIA through an example of lumber defect extraction from a lumber sample image. This is followed by an on-line extension of the MIA techniques in section 4 to monitor the prechosen defects in RGB color images of 38 lumber
samples and assign a quality grade to each sample based on extracted lumber defect information. 2. Softwood Lumber Imaging It is obvious that the results of whichever image analysis techniques are applied to grade lumber depend on the quality of the images received from the imaging sensors. To capture all possible lumber defects, one would require multiple imaging sensors that are sensitive to different regions of the electromagnetic spectrum. Gray-scale and RGB color cameras are the currently used imaging sensors in vision-based softwood lumbergrading systems.8 Ultraviolet2 and near-infrared9 (NIR) imaging spectrographs have been evaluated on test samples. The lumber samples used in this paper have been imaged using an industrial high-speed line-scan RGB digital camera system at Centre de Recherche Industrielle du Quebec (CRIQ).10 The camera acquires images of lumber boards moving on a conveyor belt at a speed of 300 ft/min (Figure 1). The scan rate of the camera between two successive line scans is 1525 Hz. For each scan, an RGB line image is acquired using a three-CCD linear array architecture behind a prismatic beam splitter to acquire separate red, green, and blue line images.11 The camera is integrated with other signal processing hardware and data processing software to form an on-line lumber board vision system. The digital signal processing (DSP) unit (Figure 1) was programmed with edge-detection and alignment algorithms to preprocess the continuous line scans acquired from the RGB camera into a lumber image prior to analysis. The preprocessing was mainly done to account for the leading and trailing ends of the lumber board, as well as to correct for lateral movement of the boards on the conveyor belt as they passed under the camera. All RGB lumber images used in this paper are the final corrected versions obtained from the CRIQ system. 3. Review of Multivariate Image Analysis This paper assumes that the reader has a basic understanding of PCA as it is applicable to MIA.1 This section provides a brief review of PCA and MIA and illustrates their use with an off-line study of defect identification from an RGB image of one lumber sample. MIA techniques, first introduced by Esbensen et al.,12 consist of extracting feature information from multi-
Ind. Eng. Chem. Res., Vol. 42, No. 21, 2003 5347
PCs have been extracted. With a multispectral image (X) each PC extracts a particular spectral feature (i.e., a unique linear combination of the pixel intensities over the spectrum of nz wavelengths). The reorganized score matrix Ta is a representation of the image in terms of that spectral feature. The above method of multiway PCA is equivalent to unfolding the three-dimensional matrix X into an extended two-dimensional matrix X and then performing ordinary PCA on it Figure 2. Different data representations of a 512 × 512 × 4 pixel multivariate image.
variate images using multiway PCA (MPCA). A multivariate image consists of a stack of congruent images, with each image in the stack representing a unique variable. Such an image can be represented as a threedimensional data set, where two dimensions (x and y) represent pixels in the image plane and the third dimension (z) represents the variable index. Figure 2 illustrates an example of a 512 × 512 pixel multivariate image with four variables, where each variable represents a different wavelength of the electromagnetic spectrum.1 The data in this multispectral image can be viewed either as a three-dimensional matrix of pixel intensities or as a two-dimensional matrix of (4 × 1 pixel) vectors at each spatial location in the (x, y) image plane, where the vectors represent the wavelength spectrum of every pixel. The variables of a multivariate image are highly correlated with each other, as they represent congruent images capturing the same pictorial information. Furthermore, multivariate images contain enormous amounts of data, making their analysis computationally intensive. To analyze such an enormous and highly correlated data set efficiently, MIA techniques rely on MPCA methods.12-14 These methods compress the highly correlated data by projecting it onto the reduceddimensional subspace defined by the dominant principal components (PCs). Multiway PCA of a three-dimensional (nx × ny × nz) digital image array X consists of decomposing it into a series of A (
