Optimal Wavelet Packets for Characterizing Surface Quality

Jan 13, 2009 - Kwanak-gu, Seoul 151-742, South Korea, and Samsung Electronics, ... wavelet packet transform is more efficient in characterizing steel...
3 downloads 0 Views 1MB Size
Download PDF
2590

Ind. Eng. Chem. Res. 2009, 48, 2590–2597

PROCESS DESIGN AND CONTROL Optimal Wavelet Packets for Characterizing Surface Quality Daeyoun Kim,† Chonghun Han,*,† and J. Jay Liu*,‡ School of Chemical and Biological Engineering, Seoul National UniVersity, San 56-1, Shillim-dong, Kwanak-gu, Seoul 151-742, South Korea, and Samsung Electronics, Myeongam-Ri 200, Tangjeong-Myeon, Asan, Chungchengnam-do 336-840, South Korea

We propose an optimal-basis texture classification strategy performed in the wavelet packet domain, in order to characterize quality-related information from a set of images. The proposed method enables one to select the discriminative texture in accordance with class information. The proposed methodology has several stages: feature extraction, feature selection, feature reduction, and classification. In the feature extraction stage, we used wavelet energy signatures obtained from wavelet packet transform. In the feature selection stage, two simple optimal-basis methods (top-down and bottom-up searching) were used to select discriminative signatures with high Fisher’s criterion values. These approaches improve classification accuracy and reduce the number of features used to classify the quality. Our proposed methodology was applied and validated to classify the surface quality of rolled steel sheets. Using this real-world industrial example, we have experimentally shown that the proposed optimal-basis approach is superior to a full-wavelet-packet-based approach, in terms of classification performance and the number of features used. 1. Introduction Texture can be described as a spatial area consisting of an arrangement of primitives resembling each other1 or an attribute representing the spatial arrangement of the gray levels of the pixels in a region of a digital image.2 In order to classify, segment, or synthesize an image, texture plays an important role in characterizing regional features.3 Texture analysis has been successively applied to many critical problems, such as object recognition, satellite photographs, computer-aided diagnosis, face recognition, and remote sensing.4 However, the classification of image data into different classes of texture has been a challenging problem in image texture analysis.1 The field of image texture analysis has not remained solely in the above-mentioned areas, but has expanded into the chemical process industry. Although more chemical process data are available online, some datassuch as that related to qualityshave still not been made available through the use of typical chemical process sensors. As digital image sensors have developed, some product surface quality can be measured by visual characteristics. Texture analysis enables one to determine the quality of product surfaces, which is very important when a product is used mainly for display or as the outer packaging of a product, such as the color and appearance of mineral flotation froth or visible patterns on the surfaces of injectionmolded plastic panels.5,6 Texture classification methods have been classified into four major categories, based on the types of features they associate with a texture: statistical, structural, model-based, and transform-based methods.3 Bharati et al.7 compared these methods and recognized that transform-based methodssespecially wavelet texture analysis, which is based on the discrete wavelet transformsexhibit the best performance in steel texture classification. Liu et al.8 confirmed that the * To whom correspondence should be addressed. Tel.: 82-2-8801887(C.H.), +82-41-535-4470(J.J.L.). E-mail: [email protected] (C.H.), [email protected] (J.J.L.). † Seoul National University. ‡ Samsung Electronics.

wavelet packet transform is more efficient in characterizing steel surfaces than the discrete wavelet transform, the latter of which was used in Bharati et al.’s work.7 Because the wavelet packet transform has bandwidth characteristics, it has a greater potential to include discriminative features for classification. Accordingly, our work uses a wavelet texture approach, based on wavelet packet transform. The wavelet packet transform uses a huge number of texture signatures, compared to other wavelet transforms. The wavelet packet transform has 4J packets, whereas the discrete wavelet transform has 3J + 1 packets at the Jth level of decomposition. These overcomplete wavelet packet bases have the advantage of including some discriminative packets and the disadvantage of having packets that are unnecessary for texture classification. Another problem is that these elements of dimensionality lead to increased computational time and costs; when using wavelet packets for bases, one needs to select the significant texture for classification. The optimal selection of a common basis for the description of a set of images remains the critical and most difficult point.9 In the signal-processing literature, there have been many feature-selection algorithms that seek to find optimal subsets by searching in an exhaustive manner.10-12 These optimal strategies are not directly applicable to texture classification. Because texture classification often treats a set of images that have many dimensions that need to be estimated, it requires a simple searching method that precludes repetitive computation. For this purpose, this approach is called the optimal-basis, bestbasis, or optimal-subband approach. Most of these algorithms have some parameters to be determined and still use recursive estimations for sorting discriminative packets. There is no rule or theoretical guideline for this, which can be a drawback in creating an automated quality-determining system. This work proposes an efficient optimal-basis methodology for texture classification, using wavelet texture analysis based on the wavelet packet transform. The proposed methodology was applied to a steel surface case study. We also used simple

10.1021/ie800536g CCC: $40.75  2009 American Chemical Society Published on Web 01/13/2009

Ind. Eng. Chem. Res., Vol. 48, No. 5, 2009 2591

Figure 1. Structures of filter bank of (a) pyramid-structured and (b) tree-structured wavelet decomposition, where A is the approximation and D is the detail section.

top-down- and bottom-up-based searching methods for selecting discriminative features, without using parameter determination or recursive searching. Some improvements are inherent in this methodology, compared to the full-wavelet packet transform methodology used by Liu et al.8 This paper is organized as follows: In section 2, the elements of wavelet texture analysis for extracting texture information from a set of images are described. Section 3 discusses the optimal-basis methodology, which is the essential component of the proposed method, with top-down and bottom-up searching. A detailed implementation of the proposed method to industrial steel sheets is provided in section 4. The results of the case study are given in section 5; in this section, the classification accuracy of the proposed optimal-basis methodology is compared with that of the previous full bases methodology, from the viewpoint of qualitative and quantitative aspects. Finally, conclusions are drawn in section 6. 2. Wavelet Texture Analysis for Texture Classification 2.1. Wavelet Transform. The principle of using wavelets is about filtering a signal through a pair of filterssa low-pass filter h and a high-pass filter gsand downsampling the filtered signals by 2 (i.e., dropping every other sample).13 A signal f(x) is decomposed into its components through shifting and dilating of a prototype function ψ(x).1 In this algorithm, a detail section is extracted at each decomposition level from the approximation section of the previous level, and the number of detail sections is reduced to one-half the number of the previous level. This is called a conventional wavelet transform or pyramid-structured wavelet decomposition8,13 (Figure 1a). The pyramid-structured wavelet transform has a set of frequency channels comprising narrower bandwidths in the lower-frequency region. In many practical cases, information related to natural textures is found not in the lower-frequency region, but rather in the middle- or higher-frequency region. These are called quasiperiodic signals.4,8 This octave frequency bandwidth of the wavelet transform is suitable for analyzing

signals whose information is found only in the low-frequency regions. Conventional wavelet transformsor pyramid-structured wavelet decompositionsis not suitable for analyzing a signal when the information is located mainly in the middle- or highfrequency regions, because of the wide bandwidth in the higherfrequency region.14 2.2. Wavelet Packet Transform. To analyze quasiperiodic signals, a generalized form of wavelet called wavelet packet basessor simply wavelet packetssis needed; it includes a library of modulated waveform orthonormal bases.1 The wavelet transform is implemented through an iterative decomposition of approximation coefficients, using a two-channel filter bank. On the other hand, a wavelet packet transform uses all of the coefficients for decomposition, resulting in an equal-frequency bandwidth. The implementation of wavelet packet bases is performed through a tree-structured filter bank, whereas the wavelet transform can be done using a pyramid-structured wavelet decomposition4,8 (see Figure 1). Each wavelet packet is estimated by relating it to some indices: a scaling parameter j, a localization parameter l, and an oscillation parameter n, where j, l ∈ Z, n ∈ N. The library of wavelet packet bases is the collection of orthonormal bases comprising functions of the form Wn(2jx - l). Wn(2jx - l) is roughly centered at 2-jl, has support of size 2-j, and oscillates n times.15 The library {Wn}n∞) 0can be obtained from a given function W0 by using the relations W2n(x) ) √2

∑ h(l) W (2 x - l) j

n

(1)

l

W2n+1(x) ) √2

∑ g(l) W (2 x - l) j

n

(2)

l

where the scaling function φ(x), W1, and the mother wavelet ψ(x) determine the function W0(x).16 The corresponding two-dimensional filter coefficients can be expressed as

2592 Ind. Eng. Chem. Res., Vol. 48, No. 5, 2009

HLL(l, j) ) h(l) h(j), HLH(l, j) ) h(l) g(j) and HHL(l, j) ) g(l) h(j), HHH(l, j) ) g(l) g(j) where the first and second subscripts denote the low-pass and high-pass filtering characteristics in the x and y directions, respectively. The two-dimensional wavelet packet basis functions can be expressed by the tensor product of two onedimensional wavelet (wavelet packet) basis functions along the horizontal and vertical planes.4,13 2.3. Wavelet Texture Analysis. Wavelet texture analysis is recognized as a powerful method, compared to other texture analysis methods used in classification.8 The number of pixels in an original image is the same as the total number of coefficients in a complete decomposition. As wavelet transform offers an orthonormal basis, its decompositions also preserve energy. The energy distribution of decompositions in a scale space can be used in texture classification.3 A wavelet texture analysis assumes that a texture has a unique distribution in the three-dimensional scale space. Two spatial axes and an additional scale axis compose the space; accordingly, if a proper discretization is performed on the scale axis of the scale space of a textured image, different textures will have different responses on the discretized scales. Expressing each subimage from the wavelet decomposition as D(J,I), where J is the depth of the level of decomposition and I is the number of packets in the depth, the energy of the subimage can be defined as ED(J,I) ) D(J, I)

| |

2 F

(3)

The normalized energy or average power is estimated from the energy divided by the number of pixels. A wavelet energy signature is vector-organized by the energies of all of the subimages. At the Jth level, the size of a feature vector for an image (when an approximation subimage is included) is 3J + 1 for the two-dimensional discrete wavelet transform and 4J for the two-dimensional full-tree wavelet packet transform.8 3. Optimal-Basis Approach Because wavelet packet bases generate an overcomplete representation, they can provide maximum discrimination power among classes in image texture analysis.1 However, a combinatorial explosion of the bases from which one can possibly choose appears because of this.8 In texture classification, a fast optimization algorithm is required to select optimal bases from whole sets of available bases. The approach to selecting features of optimal bases is part of the so-called optimal-basis paradigm or best subband strategy.17 Optimal-basis selection has two major benefits compared to the full basis approach. First, by removing indiscriminative bases, performance can be significantly improved. Using all bases does not always mean fewer classification errors. A small number of bases can have a smaller error rate than a full one.1 Second, dimension-reduction effects can result in faster analysis. The number of combinations of k bases from a total of n bases is Cnk ) n!/[k!(n-k)!]. In terms of maximum discrimination among different textures, it is not efficient to employ an exhaustive approach that determines the optimal combination by trying out each one of them.1 3.1. Optimal-Basis Selection. The problem of developing effective search strategies for feature selection algorithms has been extensively investigated in the pattern-recognition literature.11,14,18-23 However, in comparison to the use of general

feature selection strategies, the optimal-basis approach for wavelet transform is rare. As is well-known from the literature,3,4,13,19,24-26 an exhaustive or repetitive search for the global optimal solution is prohibitive from a computational viewpoint. In the case of feature selection for image texture classification, only a suboptimal solution can be attained. Some optimal-basis strategies have been applied to signal analysis. Coifman and Wickerhauser15 developed a best-basis wavelet transform that produces maximum compression. Walczak and Massart27 proposed a best-basis selection for a set of signals that selects the significant signal components. Cocchi et al.9 proposed an efficient selection algorithm based on a classification ability index. However, one cannot apply the aforementioned best-basis approach directly to texture classification problems, because of its difference from texture classification in view of the purpose, types of data sets used, and inherent computational problems. In image texture applications, the most suitable method is a local discriminate packet-based approach. Saito and Coifman17 proposed an algorithm that identifies the best-basis approach, which maximizes cross-entropy.25 However, these approaches also require recursive computation and full expansion for sorting the features and parameters to be determined. There are no theoretical rules to follow.17 3.2. Selection Criterion for Discriminate Measure. A criterion is needed to select optimal bases from a wavelet packet. What is best or optimal depends on the purpose and needs behind the methodology. For example, with signal compression needs, one must provide a good explanation of the original features, with only a few large components; with classification needs, one must derive the maximum discriminating power from among all possible bases.1 In the case of the current study, an index that represents class separation or distances among classes can be used.17 Laine and Fan3 used energy as an index for recognizing images that belong to a certain texture class.1 Bhattacharya distance28 and Karhunen-Loe´ve (K-L) transforms,29 also known as relative entropy, have been employed to reduce the number of features in a classification.19 Other indicesssuch as a similarity index, a classification error, or an information theoretical measureshave been proposed for feature selection; however, given that only a finite number of samples are available, Fisher’s criterion provides reliable class separation among the different bases. In this study, we employed Fisher’s criterion by using a trace as follows8

(

J(Wopt) ) tr

WTSBW WTSWW

)

(4)

where the between-class scatter matrix is SB and the withinclass scatter matrix is SW; the optimal projection, Wopt, is chosen as the matrix with orthonormal columns that maximizes the ratio of the between-class scatter and the within-class scatter. The objective value, denoted as J(Wopt), is called Fisher’s criterion. Among the different feature vectors, a well-classifiable one will have a high Fisher’s criterion value. We utilized these characteristics to measure a discriminative basis extracted from wavelet packet transform for the texture of images.1,8 3.3. Optimal-Basis Texture Classification Methodology. Methodologies for classifying sets of images according to texture require the satisfaction of some important conditions. Validation should be performed by using sets of images rather than a signal, multivariate discrete data, or one image. The purpose of the classification should not be solely the reduction of dimensions

Ind. Eng. Chem. Res., Vol. 48, No. 5, 2009 2593

Figure 2. Framework for optimal-basis texture classification.

like signal compression, but also the classification of multiclass textures. The chosen algorithm needs to minimize computational burdens and have procedures that are simple to implement. It is not easy to find a texture classification methodology that satisfies all of these needs. The proposed methodology improves upon previous methods by modifying some points that touch on the aforementioned issues. In feature extraction, wavelet texture analysisswhich uses signatures and not the image itself directlysreduces dimensions significantly. The wavelet packet basis approach was selected as a transform method, because of its maximum discrimination power among classes by providing equal bandwidth frequencies. The problems with overcomplete representation of full-wavelet packet bases were resolved by selecting optimal bases. Also, there is another feature-reduction step here, using Fisher’s discriminant analysis (FDA) for maximizing discrimination among classes. All of the steps of this methodology were designed to improve classification accuracy while performing the fewest computations. Figure 2 shows the optimal-basis framework for texture classification, as used in this study. The four elements involved in the framework are feature extraction, feature selection, feature reduction, and classification. The fundamental structure was taken from the machine vision framework and modified for this study.30 Two (sub)optimal searching algorithms, top-down- and bottom-up-based, were used for feature selection. A. Feature Extraction. The extraction of important textural information from images is essential for texture analysis. An image provides plenty of information that is extremely important for analysis and also has redundant or obstructive information related to texture classification. For analyzing textures, this study categorizes four methods by types of textural information: (1) statistical methods, (2) structural methods, (3) model-based methods, and (4) transform-based methods.7 Statistical texture analysis techniques consider the texture of an image as higherorder moments in its gray-scale histogram. Structural texture analysis techniques assume that texture is defined as the properties and placement rules of the texture element, such as regularly spaced parallel lines. Model-based techniques use the estimated parameters of the image models as textural feature descriptors. Transform-based texture analysis extracts textural information from an image by using the spatial frequency properties of pixel intensity variations. Bharati et al.7 compared these methods and realized that the transform-based methods especially the wavelet texture analysissoffered the best performance in terms of classifying steel textures. For texture classification, we selected a wavelet texture analysis based on

wavelet packet bases, because this has been shown to perform better than other methods.8 B. Feature Selection. We used both kinds of search methods (i.e., top-down and bottom-up) for feature selection. Feature selection was performed with a simple rule, similar to sequentialforward and sequential-backward selection. They have already been used for many pattern recognition problems, and Kumar et al. applied them to geographical analysis.19 These simple agglomerative rules do not require recursive computation for sorting discriminations of features. (1) Top-Down-Based Selection. For top-down-based selection, Copt represents the case where optimal bases consist of only selected wavelet energy signatures; CJI represents the wavelet energy signatures at level J, where I is the number of packets; CJI represents the decomposed wavelet energy signac tures of CJI; and C JI represents decomposed wavelet energy signatures of CJI. As one moves from the upper level to the lower levelsand from lower packet numbers to higher packet numbers in each levelsthe optimal bases Copt are updated by using simple selection rules, as follows: If J(CJI∪Copt) > J(CJI∪Copt), then Copt includes undecomposed signatures CJI, and if J(CJI∪Copt) e J(CJI∪Copt), then Copt includes decomposed signatures CJI, where J is Fisher’s criterion. Feature selection is performed from the upper level (level 1) to the lower level. Only the decomposed bases have higher criteria than the present levels, to be included in the optimal bases. Otherwise, undecomposed bases are included. Top-down searching requires an assumption for pruning appropriately. If J(CJI∪Copt) > J(CJI∪Copt), then J(CJI∪Copt) > J c (C JI∪Copt). It is assumed that, if the decomposition of the level 1 criterion is larger than the decomposition of level 2, then the level 1 criterion is higher than decomposed level 3. In this case, level 3 decomposition is not considered in selecting optimal bases. This precludes the need to search unnecessarily for further decomposition. (2) Bottom-Up-Based Selection. In contrast to top-down searches, bottom-up searches move from lower to upper levels. Another distinction between the two is that bottom-up searches do not require an assumption for pruning. Because a bottomup search starts at the final level, it already has a full expansion of the orthonormal bases. As one moves from lower to upper levelssand from lower packet numbers to upper packet numbers in each levelsthe bottom-up searching updates Copt by using the same simple selection rules as used in top-down searching. C. Feature Reduction. The feature-reduction step supports an efficient analysis by reducing the input dimension. The feature vectors of the original domain are converted into a smaller number of features in the new domain, along with remaining important information. In treating images, original images of several megabytes and thousands of pixels can be reduced to fewer than 100 features per image, after the extraction of textural features. Further dimension reduction can be done by using projection methods. Some popular algorithms for the purpose of dimension reduction are principal component analysis and independent component analysis, both of which perform unsupervised classification. A partial least-squares projection and Fisher’s discriminant projection can be used when class labels are available. In this study, we selected Fisher’s discriminant projection for maximizing discrimination among textural classes.

2594 Ind. Eng. Chem. Res., Vol. 48, No. 5, 2009

Figure 3. Sample images of steel sheets: (a) excellent, (b) good, (c) medium, and (d) poor quality. All original images were divided as shown in a.

D. Quality Classification. Classification algorithms can be differentiated according to the characteristics of their features. If the features as inputs have a linear correlation with each other, a simple and easy classification algorithmssuch as a linear discriminant classifierscan be used. However, when it is difficult to determine whether the features have nonlinearity, support vector machines or artificial neural networks are often preferable, as these can handle both linearity and nonlinearity by appropriately selecting model parameters such as kernel functions (i.e., support vector machine) and weights (i.e., artificial neural networks). In this study, we used K-nearest neighbors, a simple supervised classification algorithm, to classify the features. 4. Implementation for Classification of Steel Quality 4.1. Sets of Steel Images. To validate our proposed methodology, we compared optimal-basis results with previous fullwavelet packet bases in the classification of steel-surface quality. The quality of each steel surface was determined as excellent, good, medium, or poor as shown in Figure 3. The main characteristics for determining quality are the number and severity of pits on the surface. Poor-quality surfaces have several pits and show deep craters and thick, dark patterns. In comparison, excellent-quality surfaces have the fewest pits and pits that are shallow and randomly distributed. For this study, a total of 35 images of steel surfaces were used. Each image was an eight-bit gray-scale image with pixel dimensions of 479 × 508. The number of samples in each class was 8 excellent, 9 good, 6 medium, and 12 poor. Each original image was divided into four nonoverlapping (240 × 254) smaller images, as in Figure 3a. The new image set, which had a total of 140 images (32 excellent, 36 good, 24 medium, and 48 poor) was used for texture classification. The descriptions for pretreating and image acquisition are illustrated in previous works.7,8 4.2. Extraction of Wavelet Energy Signatures. We applied a two-dimensional wavelet packet transform to all 140 images, using order-one Coiflet wavelet filters up to level 4. In each level, 4J of the multiresolution coefficients were obtained, as shown in Figure 4. In full sets of wavelet packets, a total of 340 signatures were used, because full sets accumulate all of the signatures, from levels 1 to 4 (finest).

Figure 4. Decomposition levels of full-wavelet packet transform: (a) level 1, (b) level 2, (c) level 3, and (d) level 4.

Figure 5. Optimal-basis decomposition of (a) top-down- and (b) bottomup-based selection.

The choice for final decomposition (level 4) is performed in heuristics and validated by trial and error. For instance, the number of decomposition levels was determined according to certain guidelines, such that the size of the smallest subimage should be greater than 10 × 10; this criterion conforms to that of Chang and Kuo.4 We confirmed that classification accuracies with greater decomposition than that found in level 4 were significantly reduced. For feature extraction, energy signatures were calculated from subimages for each of the 140 images. Each subimage was converted to a one-scalar value. 4.3. Optimal-Basis Selection. Full-wavelet packet bases do not require feature selection and thus proceed to the next featurereduction step. As explained earlier, optimal bases were selected according to a simple selection rule. The rule searched in two directions (top-down and bottom-up), using the wavelet energy signatures as features. Top-down searching, which started from level 1 and proceeded to level 4, selected 112 signatures; bottomup searching, which performed in the opposite direction, used 154 signatures as the optimal bases (see Figure 5). 4.4. Feature Reduction and Multiclass Classification. The selected wavelet coefficients (N in number) were used as inputs for clustering. In the FDA, an N-dimensional input space is linearly projected to a three-dimensional space with a four-class case. This feature-space transformation maximizes discrimination in each class. Following FDA, we equally applied a three-nearest-neighbors classification for all bases. The misclassification rate was estimated by using a “leave-one-out” cross-validation.

Ind. Eng. Chem. Res., Vol. 48, No. 5, 2009 2595

Figure 6. Plots for full-wavelet packets of Fisher’s discriminant analysis, applied to wavelet energy signatures. Class labels are as follows: +, excellent; 4, good; O, medium; *, poor. (a) Level 1, (b) level 2, (c) level 3, (d) level 4, and (e) full sets.

5. Results and Discussion Figure 6 shows the Fisher’s discriminant plots of full-wavelet packets, including the full-sets case. As the level of decomposition increased, the distances between classes became greater. This tendency is in accordance with the results in Table 1, which showed a rise in Fisher’s criterion values as the level of decomposition increased. The level 4 plot (Figure 6d) is almost the same as that of the full sets (Figure 6e), which used all of the signatures accumulated, so the level 4 Fisher’s criterion is very similar to the full sets (Table 1). As a result, the best results obtained from full-wavelet packets were from level 4 and the full-sets case. In Figure 6, even in the level 4 and full-sets cases, some linearly inseparable points exist. The good classes and excellent

Table 1. Fisher’s Criterion Values and Numbers of Signatures Captured

level 1 level 2 level 3 level 4 full sets top-down bottom-up

Fisher’s criterion (J)

number of signatures

2.1361 3.1204 7.8397 8.8282 9.0074 27.943 53.349

4 16 64 256 340 112 154

classes are not far from each other. Figure 7shows that each class of optimal wavelet packets is more clustered and that all classes are easily separable. Fisher’s plots of two optimal bases are perfectly classifiable according to class by linear classifier.

2596 Ind. Eng. Chem. Res., Vol. 48, No. 5, 2009

Figure 7. Plots for optimal bases of Fisher’s discriminant analysis: (a) top-down- and (b) bottom-up-based selection. Table 2. Leave-One-Out Estimates (%) of Classification Errors

level 1 level 2 level 3 level 4 full sets top-down bottom-up

excellent

good

medium

poor

overall

0.125 0 0.03125 0 0 0 0

0.33333 0.16667 0.027778 0.027778 0.027778 0 0

0 0.083333 0 0 0 0 0

0.020833 0 0 0 0 0 0

0.12143 0.057143 0.014286 0.007143 0.007143 0 0

These results were validated in a quantitative manner by estimating classification errors through the use of a leave-oneout cross-validation (see Table 2). As decomposition level increased (i.e., from level 1 to level 4), full-wavelet packets showed increases in the number of signatures and decreases in the number of classification errors. Although level 4 decomposition and full sets had the fewest classification errors, they still had misclassified points in the good class. On the other hand, all of the classes of both optimal bases were perfectly classified. Table 1 compares the number of signatures used and the Fisher’s criterion for all wavelet packet bases. In full-wavelet packets, Fisher’s criterion is proportional to the number of signatures. However, in comparison with level 4 and full sets, significant increases in the Fisher’s values no longer appear to be higher than those of level 4. Optimal bases used fewer signatures (top-down, 112; bottom-up, 154) than level 4 (256) and full sets (340), showing perfect classification and large Fisher’s values. This experimentally shows that an increase in the number of signatures or features does not translate simply into an efficient classification. In Figures 6 and 7 and Tables 1 and 2, we show that our proposed method provides a more accurate classification system with fewer features than both level 4 bases and full sets of wavelet packets. Improved classification performance of optimal bases supports the fact that top-down and bottom-up searching suitably select the discriminative bases from all of the bases while using Fisher’s criterion. FDA provides another maximization of discrimination among classes. Top-down and bottom-up searching offer the same results, in terms of both Fisher’s plot and classification accuracy. However, Figure 5 shows them to be somewhat different from each other. The Fisher’s value of bottom-up searching was much higher than that of top-down searching, and top-down searching selected fewer signatures than bottom-up. Others’ work has shown bottom-up searching to have slightly better performance when using more features.3,19,25 This distinction arose from the assumption of using top-down searching for pruning. The assumptionsthat when one unde-

composed set is selected above, further decomposition is not consideredsprompts effective selection and thus precludes the need for exhaustive searching. Bottom-up searching, which is a more exhaustive approach, selects more close-to-globaloptimum values than top-down searching. These distinctions lead to different optimal-basis decompositions and Fisher’s values. 6. Conclusions Previous full-wavelet packet basis methodologies have been used widely and with remarkable performance in many texture classifications, because of their equal-bandwidth frequency characteristics. However, they also have disadvantages, including the fact that insignificant packets related to texture classification lead to misclassification. In this study, an optimal-basis texture classification methodology based on wavelet texture analysis was proposed and validated using steel-surface sheets. The selection of optimal bases was performed in two directions (i.e., bottom-up and topdown searching) and included discriminate texture using Fisher’s criterion. The performance of the proposed methodology was compared with full sets of wavelet packet bases and with each level bases. Empirical results for the four classes of steel surfaces show that this simple texture classification methodology provides improvements of performance in terms of perfect classification and a small number of features. Another benefit of this methodology is that is does not require recursive (i.e., repetitive) computation, because it has neither a sorting process for discriminate packets nor parameters that need to be predetermined. Accordingly, the proposed optimal-basis methodology will be very helpful for automated systems in determining product quality. Our proposed methodology for texture classification is applicable to any other texture classification that uses a set of multiclass images. Acknowledgment The authors thank Professor John F. MacGregor at McMaster University for his permission to use the image data for this study. The authors gratefully acknowledge partial financial support from the Seoul R&BD Program (No. R01-2004-000-10345-0); the Korea Science and Engineering Foundation, provided through the Advanced Environmental Biotechnology Research Center (R11-2003-006) at Pohang University of Science and Technology; and the Brain Korea 21 project initiated by the

Ind. Eng. Chem. Res., Vol. 48, No. 5, 2009 2597

Ministry of Education, Seoul, South Korea. This work was also supported by MKE (Ministry of Knowledge Economy) under the ETI (Energy Technology Innovation) Program (“Implementation of Advanced Energy Safety Management System for Next Generation” Project Group, Project No. 2007-M-CC23-P-051-000). Nomenclature h ) low-pass filter g ) high-pass filter f ) signal function to be analyzed j ) scaling parameter l ) localization parameter n ) oscillation parameter ∞ {Wn}n)0 ) library of orthonormal bases obtained from wavelet packet transform W0 ) given function to be transformed H ) two-dimensional filter coefficients D(J,I) ) extracted subimage at level J for the Ith packet E ) energy of subimage SB ) between-class scatter matrix SW ) within-class scatter matrix W ) weight matrix Wopt ) optimal projection weight matrix J ) objective function value or Fisher’s criterion z ) Fisher’s projection vector Cbest ) optimal bases consisting of only selected wavelet energy signatures CJI ) wavelet energy signatures at level J for I packets jCJI ) decomposed wavelet energy signatures of CJI c C JI ) decomposed wavelet energy signatures of CJI Greek Letters ψ ) mother wavelet or basis function φ ) scaling function Subscripts H ) high-pass filtering of two-dimensional wavelet packets in the given direction J ) level of decomposition I ) number of packets L ) high-pass filtering of two-dimensional wavelet packets in the given direction Operators || · ||F ) Frobenius norm tr( · ) ) trace

Literature Cited (1) Rajpoot, N. M. In Texture classification using discriminant wavelet packet subbands. In Proceedings of the IEEE 45th Midwest Symposium on Circuits and Systems (MWSCAS’2002); IEEE Press: Piscataway, NJ, 2002; Vol. 3, pp III-300-III-303. (2) Geraci, A. K. IEEE Standard Glossary of Image Processing and Pattern Recognition Terminology; IEEE Std 610.4-1990; IEEE: New York, 1990. (3) Laine, A.; Fan, J. Texture classification by wavelet packet signatures. IEEE Trans. Pattern Anal. Mach. Intell. 1993, 15 (11), 1186–1191. (4) Chang, T.; Kuo, C. C. J. Texture analysis and classification with tree-structured wavelet transform. IEEE Trans. Image Process. 1993, 2 (4), 429–441. (5) Liu, J. J.; MacGregor, J. F. Modeling and Optimization of Product Appearance: Application to Injection-Molded Plastic Panels. Ind. Eng. Chem. Res. 2005, 44 (13), 4687–4696.

(6) Liu, J. J.; MacGregor, J. F. Estimation and monitoring of product aesthetics: Application to manufacturing of engineered stone countertops. Mach. Vision Appl. 2006, 16 (6), 374–383. (7) Bharati, M. H.; Liu, J. J.; MacGregor, J. F. Image texture analysis: Methods and comparisons. Chemom. Intell. Lab. Syst. 2004, 72 (1), 57–71. (8) Liu, J. J.; Kim, D.; Han, C. Use of Wavelet Packet Transform in Characterization of Surface Quality. Ind. Eng. Chem. Res. 2007, 46 (15), 5152–5158. (9) Cocchi, M.; Seeber, R.; Ulrici, A. WPTER: Wavelet packet transform for efficient pattern recognition of signals. Chemom. Intell. Lab. Syst. 2001, 57 (2), 97–119. (10) Kalivas, J. H.; Roberts, N.; Sutter, J. M. Global optimization by simulated annealing with wavelength selection for ultraviolet-visible spectrophotometry. Anal. Chem. 1989, 61 (18), 2024–2030. (11) Jain, A.; Zongker, D. Feature selection: Evaluation, application, and small sample performance. IEEE Trans. Pattern Anal. Mach. Intell. 1997, 19 (2), 153–158. (12) Jimenez, L. O.; Landgrebe, D. A. Hyperspectral data analysis and supervised feature reduction via projection pursuit. IEEE Trans. Geosci. Remote Sens. 1999, 37 (6), 2653–2667. (13) Chang, T.; Kuo, C. C. J. A wavelet transform approach for hierarchical texture segmentation. In 1992 Conference Record of the TwentySixth Asilomar Conference on Signals, Systems and Computers; IEEE Press: Piscataway, NJ, 1992; Vol. 2, pp 816-820. (14) Alexander, E.; Nikulin, B.D. T. B. R. L. S. Near-optimal region selection for feature space reduction: Novel preprocessing methods for classifying MR spectra. NMR Biomed. 1998, 11 (4-5), 209–216. (15) Coifman, R. R.; Wickerhauser, M. V. Entropy-based algorithms for best basis selection. IEEE Trans. Inf. Theory 1992, 38 (2), 713–718. (16) Daubechies, I. Orthonormal bases of compactly supported wavelets. Commun. Pure Appl. Math 1988, 41 (7), 909–996. (17) Saito, N.; Coifman, R. R. Local discriminant bases and their applications. J. Math. Imaging Vision 1995, 5 (4), 337–358. (18) Serpico, S. B.; Bruzzone, L. A new search algorithm for feature selection in hyperspectral remote sensing images. IEEE Trans. Geosci. Remote Sens. 2001, 39 (7), 1360–1367. (19) Kumar, S.; Ghosh, J.; Crawford, M. M. Best-bases feature extraction algorithms for classification ofhyperspectral data. IEEE Trans. Geosci. Remote Sens. 2001, 39 (7), 1368–1379. (20) Vafaie, H.; De Jong, K. Robust feature selection algorithms. In Proceedings of the Fifth International Conference on Tools with Artificial Intelligence (ICTAI’93); IEEE Press: Piscataway, NJ, 1993; pp 356-363. (21) Koller, D.; Sahami, M. Toward optimal feature selection. In Proceedings of the Thirteenth International Conference on Machine Learning; Morgan Kaufmann: San Francisco, 1996; pp 284292. (22) Dash, M.; Liu, H. Feature selection for classification. Intell. Data Anal. 1997, 1 (3), 131–156. (23) Somol, P.; Pudil, P.; Kittler, J. Fast branch & bound algorithms for optimal feature selection. IEEE Trans. Pattern Anal. Mach. Intell. 2004, 26 (7), 900–912. (24) Ramchandran, K.; Vetterli, M.; Herley, C. Wavelets, subband coding, and best bases. Proc. IEEE 1996, 84 (4), 541–560. (25) Rajpoot, N. Local discriminant wavelet packet basis for texture classification. Proc. SPIE 2003, 5207, 774–783. (26) Li, D.; Pedrycz, W.; Pizzi, N. J. Fuzzy wavelet packet based feature extraction method and its application to biomedical signal classification. IEEE Trans. Biomed. Eng. 2005, 52 (6), 1132–1139. (27) Walczak, B.; Massart, D. L. Wavelet packet transform applied to a set of signals: A new approach to the best-basis selection. Chemom. Intell. Lab. Syst. 1997, 38 (1), 39–50. (28) Bhattacharyya, A. On a measure of divergence between two statistical populations defined by their probability distributions. Bull. Calcutta Math. Soc 1943, 35, 99–109. (29) Cover, T. M.; Thomas, J. A. Elements of Information Theory; Wiley-Interscience: New York, 2006. (30) Liu, J. Machine Vision for Process Industries Monitoring, Control, and Optimization of Visual Quality of Processes and Products. Ph.D. Thesis, McMaster University, Hamilton, ON, Canada, 2004.

ReceiVed for reView April 4, 2008 ReVised manuscript receiVed November 11, 2008 Accepted December 12, 2008 IE800536G