Application of Artificial Neural Networks to Rapid Data Analysis in

Article pubs.acs.org/JPCC

Application of Artificial Neural Networks to Rapid Data Analysis in Combinatorial Nanoparticle Syntheses Yuuichi Orimoto,† Kosuke Watanabe,‡ Kenichi Yamashita,† Masato Uehara,† Hiroyuki Nakamura,† Takeshi Furuya,§ and Hideaki Maeda*,†,‡,∥ †

Measurement Solution Research Center, National Institute of Advanced Industrial Science and Technology (AIST), Kyushu, 807-1, Shuku-machi, Tosu, Saga 841-0052, Japan ‡ Department of Molecular and Material Sciences, Interdisciplinary Graduate School of Engineering Sciences, Kyushu University, 6-1, Kasuga-Kouen, Kasuga, Fukuoka 816-8580, Japan § Nanotechnology Research Institute, National Institute of Advanced Industrial Science and Technology (AIST), 1-1-1, Higashi, Tsukuba, Ibaraki 305-8565, Japan ∥ Japan Science and Technology Agency (JST), CREST, 4-1-8 Hon-chou, Kawaguchi, Saitama, 332-0012, Japan ABSTRACT: The synthesis of nanomaterials is extremely sensitive to various factors under experimental conditions. Therefore, for controlling synthesis, it is important to ascertain comprehensively the relations between the conditions and nanomaterial properties. This study is intended to acquire the relations in data sets from combinatorial syntheses by means of an artificial neural network-based method toward property optimization. Recently, 3404 data sets were obtained systematically using microreactorbased combinatorial CdSe nanoparticle (NP) syntheses for examining condition− property relations. However, it is time-consuming to acquire the relations for the following reasons: (i) massiveness and complexity of the multivariate data sets, (ii) small numbers of points permitted for each experimental parameter to avoid ‘combination explosion’, and (iii) errors and missing data attributable to experimental reasons. In this work, an NN-based data analysis method was developed and applied for analyzing the data sets to acquire the relations. In the method, an exhaustive 1600 training processes and the following ensemble approach are performed for obtaining preferred NNs. Results show that NNs extract essential patterns on the condition−property relations on a realistic time scale. The trained NNs are capable of predicting the NP properties even for new experimental conditions with high accuracy. Moreover, data interpolation and sensitivity analysis based on the NNs provide us the relations as accessible descriptions such as multidimensional condition−property landscapes and key parameters for controlling the synthesis. Such information can guide us when optimizing the NP properties. Our approach is suitable to extract condition−property relations rapidly from the combinatorial synthesis data and is expected to be effective for various types of target materials, even with unknown properties, because of the flexibility of the NN analysis.

1. INTRODUCTION

powerful solution to obtain systematic experimental data, and has been of considerable practical value in the pharmaceutical industry for use in drug discovery,3 material science,4−8 and other fields. Recently, our group has performed microreactor-based combinatorial CdSe NP syntheses7,8 for which a common liquid-phase method9 was adopted. The microfluidics treatment enables us to control reaction conditions and their kinetics accurately, resulting in superior reproducibility.10−12 The target CdSe NP was selected as an appropriate test case for assessing our combinatorial system because the fundamental property of the CdSe NPs such as photoluminescence (PL) has been widely investigated for potential applications to biotags, light-emitting

Nanoscale materials have attracted considerable interest because of their potential applications to extensive fields of material science.1 On that scale, size-specific anomalous properties can be achieved by controlling the material size. For controlling nanomaterial (NM) syntheses, however, great effort is necessary, in general, because the syntheses are extremely sensitive to each factor under the experimental conditions. In addition, a rapid process for optimizing NM properties is necessary for each particular application in an industrial field. For the process, an excellent autonomous system for optimizing the nanoparticle (NP) property has been proposed in recent years.2 For that purpose, it is also effective to ascertain the relation between the experimental conditions and NM properties comprehensively through systematically performed experiments, particularly when we treat new materials with unknown properties. The approach of combinatorial chemistry3−8 is a © 2012 American Chemical Society

Received: April 1, 2012 Revised: July 10, 2012 Published: July 27, 2012 17885

dx.doi.org/10.1021/jp3031122 | J. Phys. Chem. C 2012, 116, 17885−17896

The Journal of Physical Chemistry C

Article

diodes, solar cells, and so on.1,13 Combinatorial CdSe NP syntheses with three parameters (total of 125 conditions) have been performed using the system.7 More recently, using an improved new system, combinatorial CdSe NP syntheses with six parameters have been performed at a continuous time (total of around 4500 conditions).8 As a result, we have obtained numerous and systematic data sets connecting the experimental conditions and the corresponding NP properties. Data sets from the combinatorial NP syntheses are not only numerous but also tangled and complicated on the condition− property relations. Furthermore, the liquid-phase method is normally controlled by many experimental parameters. In the syntheses, therefore, data points for each experimental parameter are limited to a small number to avoid ‘combination explosion’, although the whole number of experiments is numerous. The great distance between adjacent conditions often results in ambiguous understanding of the property trends. In addition, errors and missing data attributable to experimental reasons also engender unclear understanding of the trends. For these reasons, it is time-consuming to acquire condition−property relations from data sets. The concepts of data-mining and informatics have been developed to obtain valuable information from complicated and numerous data.5,6,14,15 Artificial neural networks (ANNs) are well-systemized nonlinear data analysis methods that have been widely applied to data analysis in various fields of research.16−20 The NN can flexibly analyze various types of data and extract important patterns related to input−output relations in the data. Consequently, well-trained NNs can predict outputs even for new inputs. As explained in this article, an NN-based data analysis method was developed and applied for analyzing the thousands of data sets from the combinatorial NP syntheses with six parameters. We examined the accuracy and efficiency of our method for acquiring condition−property relations in the data sets. Moreover, valuable results from the analyses are presented. Finally, we discuss the effectiveness of the method for a rapid data analysis process for the combinatorial NM syntheses toward property optimization.

Figure 1. Concept of ANN-based data analysis for combinatorial nanomaterial syntheses.

overtraining situation, in which NNs can predict training data sets, whereas they show low prediction ability for new data. To ascertain the generalization ability of the NN, we checked the prediction ability for new data after the training step. The NNs with high generalization ability can be regarded as a ‘knowledge assembly’ on the multidimensional condition−property (X−Y) relations. Valuable information for optimizing NM properties can be extracted from the assembly on demand (panel c). Experimental results for new experimental conditions can be predicted within the interpolation range. The NN-based data interpolation provides condition−property (X−Y) and property−property (Y−Y) landscapes as understandable 3D graphs and so on. These graphs help us to find the trends and limitations of the condition−property (and property−property) relations. In addition, an NN-based sensitivity analysis (SA) can reveal key experimental parameter(s) for controlling the specific NP property. The key of our concept is the success in producing NNs with high generalization ability in panel b on a practical time scale by analyzing numerous data sets from combinatorial syntheses.

2. CONCEPT OF ANN-BASED DATA ANALYSIS FOR COMBINATORIAL NANOMATERIAL SYNTHESIS In this section, we will describe the concept of the NN-based data analysis for combinatorial NM synthesis. (See Figure 1.) Here n experimental parameters and m different points for each parameter are assumed for the combinatorial syntheses. In addition, k NM properties are considered. According to the assumptions, multivariate condition−property (input−output or X−Y) data sets Di(X1i, X2i..., Xni, Y1i, Y2i..., Yki) (i = 1, 2..., mn) are obtainable after the syntheses (panel a). Data analysis starts from a NN training step using a part of the data sets (panel b). It is unnecessary to assume any model function in advance for each condition−property relation. In addition, all possible input and output factors are useful irrespective of their importance. Noise and errors in the data sets would not be trained under appropriate training conditions. The flexibility of the NN analysis for various types of data derives from these features and can contribute to a rapid data analysis process. Appropriately trained NNs are capable of predicting all experimental data including new data. By this situation, it is said that the NNs have high generalization ability. Such NN includes the important patterns on condition−property (X−Y) relations. In contrast, undesirable training conditions often cause an

3. DATA SETS FROM COMBINATORIAL NANOPARTICLE SYNTHESIS Combinatorial CdSe NP syntheses8 were performed by the system shown at the bottom of Table 1. The system consists of pump (P1−P4), mixing, reaction, detection, and PC controller parts. The mixture solution of raw materials was heated at the reaction part, and the ultraviolet−visible (UV−vis) absorption and PL of the solution were measured online. 17886



Article

Table 1. Experimental Parameters for Combinatorial CdSe Nanoparticle Syntheses Using Three Amine Species: Dodecylamine (DDA), Oleylamine (OLA), and n-Octylamine (OA)

a

7, 15, and 30 s are only adopted for the reaction time. b2, 3, 7, and 15 s are only adopted for the reaction time. c2 and 3 s are only adopted for the reaction time. d3, 7, 15, and 30 s are only adopted for the reaction time. e2, 3, and 7 s are only adopted for the reaction time.

4. METHODOLOGICAL DETAILS ON NN-BASED DATA ANALYSIS The commercial neural network software NeuralWorks Predict21 was used as the NN computational engine for our method. This software is implemented on the Microsoft Excel (version 2002− 2003). To perform automatic and exhaustive NN analysis, an Excel VBA program was developed based on Predict’s command line user interface. All NN computations were performed on a desktop personal computer: NEC Mate-J ME-7 (Intel Core2 Quad CPU Q9550 [2.83 GHz], 4 GB RAM) with Microsoft Windows XP Professional version 2002, SP3. 4.1. Neural Network Architecture. Figure 2 shows a feedforward type NN constructed using Predict software. The NN consists of input, hidden, and output layers; in addition, processing elements (PEs) are included in each layer. The hidden layer does not form an obvious layer architecture. The PEs can be linked between adjacent layers (solid lines in the Figure), between input and output layers (dashed-dotted lines), and between PEs in the hidden layer (broken lines). The cascade-correlation learning algorithm22 is adopted for determining the network architecture in Predict. The training process under the algorithm starts from the simplest network structure with no hidden layer. The network changes automatically into a more complicated one including the hidden layer as needed according to the NN performance. For each input variable, not only a linear PE but also various transformed PEs are generated and selected automatically using a genetic algorithm to optimize the NN performance. Consequently, more than one PE can be selected for each input variable. In the input layer, furthermore, values with a real world

Table 1 presents experimental parameters and their values (data points) for each parameter considered in the combinatorial syntheses. Variant parameters, (i) and (ii), were reaction temperature, reaction time, amine concentration, standard Cd concentration, and Cd/Se ratio. The standard Cd concentration at point C (see the system illustration) is defined as the Cd concentration when the P1/P2 flow rate at point A is assumed to be 1. For the NN analysis, however, we used an actual Cd concentration determined by the actual P1/P2 flow rate. Constant parameters in (iii) were (oleic acid)/Cd and TOP/ Se ratios. Three amine species were used as additive reagents: ndodecylamine (DDA), oleylamine (OLA), and n-octylamine (OA). The combinatorial syntheses are controlled using six variant parameters by counting the kinds of amine species as a parameter. For each amine species, 1500 conditions can be extracted from the combinations of the experimental parameters. Consequently, the total sum of the conditions is 4500. During the syntheses, the UV−vis and PL spectrum data were analyzed automatically using our developed spectrum peakanalysis program. The analysis provides the wavelength (WL), peak intensity, full width at half-maximum (fwhm), and peak area of the main peak. On the basis of the results, particle size, CdSe reaction yield (RY), PL quantum yield (QY), and so on can be estimated.7,8 Consequently, we obtained both the UV−Vis and PL spectra for 1324, 929, and 1151 data sets for DDA, OLA, and OA species, respectively (a total of 3404 data sets). In this work, our NN-based method has been applied for analyzing the data sets for each amine species separately as the first step. However, all data sets can be analyzed together by translating the kinds of amine species into one property value. 17887



Article

prediction ability for the selection data sets. NNs with high generalization ability are selected by the ranking. (3) Validation data sets are prepared to check the generalization ability of the selected NNs. It is noteworthy that the validation data sets are used neither for training nor for selecting NNs. (4) Training{inside} data sets are a part of the training data sets and are actually used for NN training inside Predict. (5) Test{inside} data sets are the remaining part of the training data sets. The test{inside} data sets are used for checking the generalization ability inside Predict to avoid an overtraining situation during the training step. NNs with various features are obtainable by changing the training conditions. In normal use of Predict, a training condition is determined by four training parameters, that is, parameters on a noise level, data transformation level, variable selection level, and neural network search level. These training parameters are explainable as follows:21 (i) Noise level is related to how Predict avoids overtraining situations against the noise in the data. Candidate levels of four types are prepared for the parameter: {clean data}, {moderately noise data}, {noisy data}, and {very noisy data}. (ii) Data transformation level is related to how Predict searches data transformations to achieve suitable data distribution leading to preferable NNs. Four candidate levels are prepared: {scale data only}, {superficial data transformation}, {moderate data transformation}, and {comprehensive data transformation}. (iii) Variable selection level is related to how Predict searches better input variables from original and a series of transformed input variables. Five candidate levels are prepared: {no variable selection}, {superficial variable selection}, {moderate variable selection}, {comprehensive variable selection}, and {exhaustive variable selection}. (iv) Neural network search level is related to how Predict searches the preferable network. Five candidate levels are prepared: {no network search}, {superficial network search}, {moderate network search}, {comprehensive network search}, and {exhaustive network search}. However, {no network search} was not used in this work because it does not construct NNs. The Predict user chooses a level for each training parameter. To search exhaustively for suitable training condition(s), 320 different conditions from all combinations among the candidate levels are used for NN construction. In this work, we conducted a wide-range search of preferable NNs beyond the 320 training conditions (see Figure 3). In addition to the training conditions, the constructed NNs are affected by starting parameter sets given for the PEs and links in the network (designated as random seed (RS)) (see also Figure 2). Five different starting parameter sets, (RS)L, L = 1−5, were used for the wide-range search. In all, 1600 (= 5 × 320) NNs were obtained after the training step for the identical training data sets. 4.3. Ranking of NNs and Ensemble Approach. To evaluate the prediction ability of the NNs, we used the Pearson product-moment correlation coefficient. The correlation coefficient between experimental values y(u) I,i (i = 1, 2, 3, ..., nα) and NN predicted values yNN,(u) (i = 1, 2, 3, ..., nα) is defined for Ith I,i NP property (YI) as

Figure 2. Procedures of NN training based on Predict software for analyzing data sets from combinatorial syntheses.

unit are converted to those with a network unit by application of normalization and centering for all the input variables. An inverse transformation from a network unit to a real world unit is conducted at the output layer. For this work, we adopted multi-input single-output NNs as presented in Figure 2 to avoid worsening of the prediction ability with increasing network size. Although we use the single-output network, more than one NM property can be optimized simultaneously without losing the NN prediction ability by the following strategy. First, a single-output NN is constructed separately for each property, and data interpolation is conducted based on the NN. Next, a preferable condition area is determined for each property according to the interpolated results. Finally, the optimum condition(s) for more than one property can be determined by finding the overlap among the preferred areas determined separately. 4.2. Constructions of Neural Networks (Data Partitioning and NN Training). The procedures for constructing NNs are exhibited in Figure 2. Two steps of data partitioning are performed initially. In the first partitioning, the data sets are divided randomly into three parts, training, selection, and validation data sets outside Predict. To improve data sampling, we use a common stratified random sampling for the first data partition. The specific percentages of the data sets are selected randomly from each histogram’s bin for the training (50%), selection (40%), and validation (10%) data sets. In the second partitioning, the training data sets were divided further into two parts, training{inside} (70% of the training data sets) and test{inside} (30% of the training data sets) data sets inside Predict according to the software default setting. The five kinds of data sets are explained as shown below. (1) Training data sets are used for constructing NNs. Then, they are further divided into training{inside} and test{inside} data sets in Predict. (2) Selection data sets are used for selecting preferable NNs. The constructed NNs are ranked according to their 17888



Article

Figure 3. Procedures for a wide-range search of the optimum NNs and the following ensemble treatment.

1600), according to the FNN,(net) I,{selection} value, where the NN1 and NN1600 respectively represent the best and worst NNs on the ability. In this work, an ensemble method19 was applied to obtain more preferable NNs. An ensemble neural network (ENN) can be described as the combination of a finite number of the component NNs (CNNs). In general, the ensemble approach can provide a robust (stable) NN with higher generalization ability compared with each component network. We adopted the simplest ENN using the numerical average (see also Figure 3c) defined as

,(u) RINN ,{α} = n

,(u) ∑i =α 1 (yI(,ui) − yI̅ (u) )(yINN − yI̅ NN ,(u) ) ,i n

n

,(u) ∑i =α 1 (yI(,ui) − yI̅ (u) )2 ·∑i =α 1 (yINN − yI̅ NN ,(u) )2 ,i

(1)

where {α} stands for the kind of data set. Subscript i signifies the ith data set in data set {α}, and nα denotes the total number of {α}. Superscript (u) means the unit of the data sets. Some examples are that y(net) and y(real) values are treated, respectively, I,i I,i in the unit of the network and real world. The y(u) and yNN,(u) , I̅ I̅ (u) NN,(u) respectively, represent the averages of the yI,i and yI,i values in {α}. For convenience, the prediction ability is also evaluated as ,(u) ,(u) FINN = 1 − RINN ,{α} ,{α}

(real) yIENN ,j

NIENN

=

∑

K (real) yICNN ,j

K =1

(2)

(3)

K(real) where the yCNN represents the output of I,j ranking. The NENN is the number of CNNs I

the Kth NN in the considered for the ensemble. To determine the appropriate number NENN I,best , we ENN compare the FENN,(real) . I,{selection} values among different NI 4.4. Confirmation of NN Generalization Ability. Finally, we examined the prediction ability for the validation data sets, ENN best,(real) RI,{validation} , to confirm the generalization ability of the determined ENN. We also introduced the mean-absolute-error (MAE) defined as

The value of FNN,(u) I,{α} approaches zero when the NN in question shows high prediction ability. The other evaluation functions used in the present work will be introduced as needed. NN,(net) During the training step, RI,{test{inside}} is used to avoid overtraining in Predict. For all the 1,600 NNs after the training, the prediction ability for the selection data sets is examined using FNN,(net) I,{selection} (see Figure 3b). Consequently, we can obtain the ranking list on the generalization ability, NNK (K = 1, 2, 3, ..., 17889



Article

Figure 4. Dependence of the number of component NNs (NENN) on the prediction ability of the ENN for selection data sets for dodecylamine (DDA): (a) overall view and (b) the corresponding enlarged view. The NENN,best (same as the NbestENN) is the number providing the smallest F{selection}ENN,(real) in each property. best ,(real) MAEIENN = ,{α}

nα ) − yIENN ∑ (|yI(,real ,i i

best ,(real)

i=1

Table 2. Prediction Ability of the Constructed ENNs for Nanoparticle Properties

|)/nα (4)

where nα denotes the number of data sets {α}. All procedures in Section 4 can be performed automatically by our developed Excel-VBA-based program.

(a) DDA

5. RESULTS AND DISCUSSION We specifically examined three points related to the effectiveness of our method as follows: (i) prediction ability of the constructed NN, (ii) efficiency of the method, and (iii) fruits from the NN analysis of the condition−property relations. 5.1. Number of Component NNs for Constructing ENNs. Figure 4 shows the prediction ability of the constructed ENN ENNs, FENN,(real) . Results I,{selection}, for DDA as a function of the NI showed that all of the curves for the four NP properties show a minimal value at the small number of NENN . The NENN I I,best providing ENN,(real) the smallest FI,{selection} are 11, 14, 11, and 8, respectively, for PL− WL, PL−QY, PL−fwhm, and CdSe−RY. Moreover, the ENN FENN,(real) = 40 and 130 I,{selection} increases monotonically between NI ENN for every curve. The optimal NI,best is expected to be determined as a balance between effects of two types. One is the ensemble effects by which more than one network compensates their weak prediction area each other. The other is a contamination of inferior networks to the ENN. The NENN I,best results for all of the amine species are presented in Table 2. The FENN,(real) I,{selection} curves for OLA and OA show a similar trend to those for DDA. In every case, the results show ensemble ENN effects (NENN I,best > 1) and the different NI,best from each other. The ENN determined NI,best is used for constructing ENNs in the remaining part of this article. 5.2. Generalization Ability of Constructed ENNs. Figure 5 presents relations between the experimentally measured values and the ENN-based predicted values (see also Table 2 showing best,(real) best,(real) the RENN and MAEENN results). Figure 5c−f for DDA, I,{α} I,{α} respectively, shows the prediction ability for the validation data sets for WL, QY, fwhm, and RY. Panel c for the WL shows an excellent linear relation between the measured and the predicted ENNbest,(real) values (RI,{validation} = ca. 0.98). The graphs for the QY and RY ENNbest,(real) (panels d and f) also show good linearity (RI,{validation} ≥0.92 for both cases). In addition, the measured−prediction difference increases with the values because of the mathematical reason on ENNbest,(real) the percentage calculations. The RI,{validation} (= ca. 0.70) for the

PL-QY

PL-fwhm

CdSeRY

11

14

11

8

,(real) R {ENN training }

0.983

0.926

0.776

0.969

,(real) R {ENN selection}

0.977

0.888

0.759

0.961

,(real) R {ENN validation}

0.977

0.920

0.702

0.937

,(real) ENN ,(real) R {ENN selection} / R {training }

0.99

0.96

0.98

0.99

,(real) ENN ,(real) R {ENN validation} / R {training }

0.99

0.99

0.90

0.97

,(real) MAE{ENN training }

3.5 nm

3.3%

2.2 nm

3.4%

,(real) MAE{ENN selection}

4.2 nm

4.1%

2.4 nm

3.7%

,(real) MAE{ENN validation}

4.2 nm

4.0%

2.5 nm

3.9%

5

7

(b) OLA

ENN Nbest

12

21


0.961

0.898

0.797

0.948


0.948

0.885

0.725

0.951


0.932

0.857

0.631

0.952


0.99

0.99

0.91

1.00


0.97

0.95

0.79

1.00


4.3 nm

2.4%

2.1 nm

4.9%


5.0 nm

3.2%

2.4 nm

4.8%


5.7 nm

3.3%

2.5 nm

5.1%

ENN Nbest

6

4


0.916

0.921

0.781

0.910


0.932

0.895

0.846

0.887


0.926

0.920

0.706

0.918


1.02

0.97

1.08

0.97


1.01

1.00

0.90

1.01


5.3 nm

1.6%

2.7 nm

5.6%


6.0 nm

1.9%

2.7 nm

6.1%


6.0 nm

1.7%

3.1 nm

5.1%

(c) OA

17890

ENN Nbest

PL-WL

34

7



Article

Figure 5. Prediction ability of the constructed ENNs for nanoparticle properties (PL−WL, PL−QY, PL−fwhm, and CdSe−RY) for dodecylamine (DDA) [(a)−(f)], oleylamine (OLA) [(g)−(i)], and n-octylamine (OA) [(j)−(l)]. The figures include predictions for (a) training, (b) selection, and (c−l) validation data sets. Panels i and l, respectively, depict enlarged views of panels h and k. Subscripts and superscripts of the R and MAE are partially omitted because of space limitations.

validation data sets were selected randomly from multidimensional data sets. Therefore, the accurate prediction for the validation data sets implies that the constructed ENNs acquired multidimensional condition−property relations in the data. Panels a and b (for WL of DDA), respectively, show the ENN prediction ability for the training and selection data sets. In ENNbest,(real) best,(real) addition, Table 2 presents the ratios of RENN {selection} /R{training} ENNbest,(real) ENNbest,(real) and R{validation} /R{training} . The deviation of these ratios from 1 is less than or equal to 5% except for the fwhm. The greatest deviation is 21% for the fwhm of OLA. Therefore, the ENNs have similar prediction ability irrespective of the kinds of data sets, that

fwhm (panel e) is lower than that of the other NP properties because the range of the fwhm (ca. 35−55 nm) is comparable to best,(real) the magnitude of the error (MAEENN I,{validation} = 2.5 nm), leading to best,(real) the degradation of the linearity. In contrast, the MAEENN I,{validation} for fwhm is sufficiently small to predict the property. Figure 5g−l shows the measured−prediction relation for the validation data sets for the OLA and OA (see also Table 2). For best,(real) OLA and OA, almost all RENN I,{validation} values show more than 0.86 ENNbest,(real) except for the fwhm (RI,{validation} > ca. 0.63). However, good prediction accuracy can be confirmed for the fwhm using ENNbest,(real) ( 40% covers 7−30 s. Panels b−d correspond to the surfaces of the other NP properties under the same condition as panel a. In panel b, the WL increases monotonically with the increment of reaction time and temperature in region A. Panel c shows that the region A with high QY corresponds to the bottom of the well on the fwhm surface. In panel d, the RY monotonically increases concomitantly with increasing the temperature, although the RY has a local maximum point at around 7−15 s for the reaction time. Comparison of region A between panels a and d implies that the RY can be increased by ∼10% without decreasing QY by shortening the reaction time from 30 s corresponding to the highest QY. Panels e and f are QY graphs involving the amine concentration as an axis under the condition with the highest QY. By focusing the amine concentration, the QY has a local maximum value at ∼10 wt % over the wide range of temperature (panel e) and reaction time (panel f). The surfaces in Figure 7 guide us for designing NPs with more than one desired

we used all of the combination of the experimental parameters in Table 1, that is, a total of 1500 points of experimental conditions for DDA, for example. Figure 6 presents results of the SA for DDA. Panels a−d, respectively, show mean-square values of the sensitivity for WL, QY, fwhm, and RY. These graphs provide information related to which experimental parameter is an important or sensitive factor for the specific NP property. For example, panel c showed that the reaction time has a strong correlation with the fwhm. Therefore, the reaction time should be treated carefully for controlling the fwhm. From panel d, the correlation between the amine concentration and RY was found to be dominant. By summing the sensitivity only over the specific reaction temperature, one can examine the sensitivity on the specific temperature in question. Figure 6e,f presents results for WL of DDA. Panel e (average value) shows that the reaction time has strong correlation with WL, and the magnitude of the correlation decreases concomitantly with increasing the temperature. In contrast, the correlation of the Cd/Se ratio increases as the temperature increases. In panel f (mean square value), results showed that the magnitude of the sensitivity of reaction time depends strongly on the temperature. In particular, the reaction time is dominant at ∼240 °C, which means that we must notice the reaction time at around the temperature. As shown above, the NN-based SA provides advice for controlling syntheses with smaller errors and can also contribute to instrument developments. 5.5. NN Data Interpolation for Describing Multidimensional Condition−property Landscapes. We performed data interpolation using the ENNs to acquire the condition− property relations. In this work, interpolation is conducted on 17893



Article

Figure 8. Multidimensional expression of condition−property (QY) landscapes based on ENN data interpolation for DDA.

properties and also tell us the limitation on the condition− property relation. Figure 8 is a 6D expression for the relation between the QY and five experimental parameters described based on the 3D graphs. This expression helps us to understand the trends of the QY for the five parameters simultaneously and guides us to control the NP property. By considering the DDA concentration, the undulation of the QY surface is prominent at the 10 wt % at every case in the Figure. The undulation is enlarged by increasing the Cd/Se ratios of 0.2 to 0.8. In addition, the undulation is enlarged gradually by the increment of the Cd concentration (4 mM → 16 mM); the highest QY value on the surface is elevated by the increment. Figure 8 provides us valuable advice about ‘what amount’ and ‘which direction’ we should change each experimental parameter to optimize the NP property. Figure 9 shows the 3D scatter plots on the property−property relations using ENN data interpolation; the total of 26 460 data points predicted by the interpolation was included in each graph. Panel a is described as a function of the predicted WL and fwhm, whereas panel c is as a function of the WL and RY. These data points were colored according to the reaction temperature. The RY and fwhm values are regarded as the size of the plots in the panels a and b, respectively. Here larger plots correspond to larger RY or fwhm values. Therefore, each panel expresses 5D

information, that is, three NP properties (by plot position), temperature (by plot color), and one additional NP property (by plot size). Panels c and d, respectively, show the projected plots of the data points in panels a and b. In these panels, the plot size is not considered for the sake of convenience. Panels a and c show that the QY data points distribute as a trigonal pyramidal form that tilts to the corner with long WL and small fwhm sides. Data points with QY > 40% concentrate at the region of long WL (around 550 nm and more) and small fwhm (around 35 nm). The increment of temperature gradually spreads the QY distribution toward the range with longer WL and smaller fwhm. In addition, the data points for ∼270 °C dominate the area of high QY value. Regarding on the RY (plot size), basically, the RY increases gradually with the increment of WL except for the points with higher RY appear at ∼550 nm of the WL. Panels b and d exhibit that the QY data points distribute as a broad shape on the WL−QY plane. Data with QY > 40% concentrate at the region with long WL and the whole range of RY. The increment of temperature (plot color) shifts the WL and RY values toward longer and higher areas, respectively. Figure 9 can accelerate the acquisitions of the trends and limitations on the property−property relations. It was confirmed that our NN-based method is effective for analyzing the numerous data sets from the combinatorial NP 17894



Article

Figure 9. Three-dimensional scatter plots of the predicted QY after ENN data interpolation for DDA expressed as a function of (a) predicted WL and fwhm and (b) predicted WL and RY; here each panel shows two different views (views A and B). Reaction temperature and one additional property (RY for (a) and fwhm for (b)) are expressed, respectively, by the color and size of plots. Panels c and d are projected plots corresponding to panels a and b, respectively; the plot size is not considered in panels c and d.

synthesis to acquire the condition−property relations. The prediction ability of the NNs can cover the shortness of the data points for each parameter in the synthesis. The NN-based data interpolation and SA can provide understandable expressions on the relations and guide us to optimize NP properties. According to these guides, one can make a more effective strategy for obtaining NPs with desired properties for various application needs, particularly when we treat new materials with unknown properties. As previously mentioned, the NNs guide us to optimize NP properties from various directions in our concept. From that standpoint, our approach is different from a method specialized only for the prediction of optimum condition(s). If we want to predict accurately the optimum condition, then a sufficient number of data sets is required around the condition to avoid the overtraining situation. However, such ideal data distribution is not promised in general, in particular, in the syntheses for new materials. In our approach, therefore, the

accuracy of the predicted optimum condition strongly depends on the data distribution, and the optimum condition should be considered as only one (but important) of the guides. The accurate prediction of the optimum conditions is a meaningful branch of the research. It also should be considered as an important target in our future work.

6. CONCLUSIONS In this work, the ANN-based method was applied for analyzing a total of 3404 data sets from combinatorial CdSe NPs syntheses with six parameters to acquire the condition−property relations on the data sets. The NN analysis was performed for the data sets for each amine species separately, that is, DDA (1,324 data sets), OLA (929 data sets), and OA (1151 data sets). Our method includes exhaustive training and ensemble processes for searching superior NNs with higher generalization ability. We trained 1600 different NNs and constructed an ensemble NN 17895



Article

(11) Chan, E. M.; Mathies, R. A.; Alivisatos, A. P. Nano Lett. 2003, 3, 199−201. (12) Krishnadasan, S.; Tovilla, J.; Vilar, R.; deMello, A. J.; deMello, J. C. J. Mater. Chem. 2004, 14, 2655−2660. (13) Parak, W. J.; Gerion, D.; Pellegrino, T.; Zanchet, D.; Micheel, C.; Williams, S. C.; Boudreau, R.; Le Gros, M. A.; Larabell, C. A.; Alivisatos, A. P. Nanotechnology 2003, 14, R15−R27. (14) Manly, C. J.; Louise-May, S.; Hammer, J. D. Drug Discovery Today 2001, 6, 1101−1110. (15) Rothenberg, G. Catal. Today 2008, 137, 2−10. (16) Burns, J. A.; Whitesides, G. M. Chem. Rev. 1993, 93, 2583−2601. (17) Sun, Y.; Peng, Y.; Chen, Y.; Shukla, A. J. Adv. Drug. Delivery Rev. 2003, 55, 1201−1215. (18) Huesken, D.; Lange, J.; Mickanin, C.; Weiler, J.; Asselbergs, F.; Warner, J.; Meloon, B.; Engel, S.; Rosenberg, A.; Cohen, D.; Labow, M.; Reinhardt, M.; Natt, F.; Hall, J. Nat. Biotechnol. 2005, 23, 995−1001. (19) For example: (a) Maqsood, I.; Khan, M. R.; Abraham, A. Neural Comput. Appl. 2004, 13, 112−122. (b) Zhao, Z.; Zhang, Y.; Liao, H. Eng. Appl. Artif. Intell. 2008, 21, 1182−1188. (20) For example: Szecówka, P. M.; Szczurek, A.; Licznerski, B. W. Sens. Actuators, B 2011, 157, 298−303. (21) NeuralWorks Predict, version 3.24; NeuralWare, Inc.: Carnegie, PA, 2009. (22) Fahlman, S. E.; Lebiere, C. The Cascade-Correlation Learning Architecture. In Advances in Neural Information Processing Systems; Touretzky, D., Ed.; Morgan-Kaufman: San Mateo, CA, 1990; Vol. 2, pp 524−532.

(ENN) based on the component NNs. It takes about 2.5−5.0 h in real time to construct 1600 NNs on a desktop computer. Results showed that the constructed ENNs can well-predict the NP properties even for new experimental conditions. For example, correlation coefficients for the validation data sets were found to be R = 0.977 for the PL−WL of DDA. Therefore, the NN acquired important patterns on the condition−property relations in the data sets. The prediction ability of the NNs for new experimental conditions (generalization ability) can cover the small number of data points for each parameter in the combinatorial synthesis. Results show that the NN-based data interpolation provides us multidimensional condition−property landscapes as accessible 3D graphs, for example. In addition, the NN-based SA can extract key experimental parameter(s) for controlling the NP property from the condition−property relations. This information guides us to control the synthesis for the optimization of the NP properties. It can be concluded that our NN-based analysis is a suitable method for analyzing numerous and complicated data sets from combinatorial NM syntheses to obtain the comprehensive condition−property relations. In particular, the analysis is effective for new materials with unknown properties because of the flexibility of NN. The condition−property trends acquired from the analysis make it possible to construct an effective strategy for obtaining NMs with more than one desired property. We believe that this approach accelerates the material design process for various application needs in the field of material science.

■

AUTHOR INFORMATION

Corresponding Author

*Tel: +81-942-81-3676. Fax: +81-942-81-3657. E-mail: [email protected]. Notes

The authors declare no competing financial interest.

■

ACKNOWLEDGMENTS This research was supported by JST, CREST. All computations for the artificial neural network based data analysis were performed using PCs in our laboratory.

■

REFERENCES

(1) For example: Park, J.; Joo, J.; Kwon, S. G.; Jang, Y.; Hyeon, T. Angew. Chem., Int. Ed. 2007, 46, 4630−4660. (2) Krishnadasan, S.; Brown, R. J. C.; deMello, A. J.; deMello, J. C. Lab Chip 2007, 7, 1434−1441. (3) Hijfte, L. V.; Marciniak, G.; Froloff, N. J. Chromatogr., B 1999, 725, 3−15. (4) Cawse, J. N. Acc. Chem. Res. 2001, 34, 213−221. (5) Maier, W. F.; Stöwe, K.; Sieg, S. Angew. Chem., Int. Ed. 2007, 46, 6016−6067. (6) Rajan, K. Annu. Rev. Mater. Res. 2008, 38, 299−322. (7) Toyota, A.; Nakamura, H.; Ozono, H.; Yamashita, K.; Uehara, M.; Maeda, H. J. Phys. Chem. C 2010, 114, 7527−7534. (8) Watanabe, K.; Orimoto, Y.; Nagano, K.; Yamashita, K.; Uehara, M.; Nakamura, H.; Furuya, T.; Maeda, H. Chem. Eng. Sci. 2012, 75, 292− 297. (9) Murray, C. B.; Norris, D. J.; Bawendi, M. G. J. Am. Chem. Soc. 1993, 115, 8706−8715. (10) (a) Nakamura, H.; Yamaguchi, Y.; Miyazaki, M.; Maeda, H.; Uehara, M.; Mulvaney, P. Chem. Commun. 2002, 2844−2845. (b) Nakamura, H.; Tashiro, A.; Yamaguchi, Y.; Miyazaki, M.; Watari, T.; Shimizu, H.; Maeda, H. Lab Chip 2004, 4, 237−240. 17896


Application of Artificial Neural Networks to Rapid Data Analysis in

Recommend Documents