4160
Ind. Eng. Chem. Res. 2009, 48, 4160–4164
Neural Network Prediction of Interfacial Tension at Crystal/Solution Interface K. Vasanth Kumar* Departmento de Engenharia Quıymica, Faculdade de Engenharia da UniVersidade do Porto, Rua Dr. Roberto Frias, 4200-465 Porto, Portugal
Interfacial tension at the crystal/liquid interface is a crucial and important parameter in crystal growth kinetics. The objective of the present study is to develop a neural network that is simple to use for predicting this important parameter using only from the information of solubility, molecular weight, and density of the studied systems. A three-layer feed-forward neural network was constructed and tested to predict the interfacial tension at the crystal/solution interface. The concentration of solute in liquid phase, concentration of solute in solid phase, temperature, density and molecular weight of crystal were used as inputs to predict the interfacial tension at the crystal/liquid interface (σSL). The network was trained using the solubility information for 28 systems to predict the σSL value and was validated with 29 new systems. Despite the limited number of data used for training, the neural network was capable of predicting σSL successfully for the new inputs, which are kept unaware during the training process. The σSL value that is predicted by the artificial neural network during the training and testing process was compared with σSL predicted from the widely used empirical expression. For most of the systems, ANN better predicts σSL, when compared to empirical correlation. 1. Introduction The surface energy of sucrose has an important role in the kinetics of growing crystals in pure solutions. The surface property can explain how the solute particles in solution interact with the crystal surface. The surface property also helps to understand how crystals interact with themselves during the growth process. The growth of crystals in solution is greatly dependent on the fluidity of the mixture.1 The interfacial energy also has an important role in the stability of colloidal suspensions and adsorption of molecules onto solid surfaces and thereby play crucial role on the mass-transfer process.2 The relationship between the interfacial tension at the crystal/liquid interface (σSL) of the solid crystal and that of the surrounding saturated solutions can be determined from the classical nucleation theory.2,3 Several empirical correlations have been developed so far to determine the interfacial tensions from physical properties that can be directly determined. Digilov4 presented an empirical correlation that related the melting point with the interfacial tension at the crystal/melt interface. Empirical expressions correlating the solubility and interfacial tension at the crystal/ solution interface were reported by So¨hnel5 for sparingly and readily soluble crystals. Many empirical expressions that correlated solubility with interfacial tension were reported by Bennema and So¨hnel,6 Nielson and Sohnel,7 and Mersmann8 for crystal/solution systems at equilibrium conditions. Sangwal9 reported the relationship between the surface tension and the mole fraction of solute in solution. Of these available expressions, the Mersmann expression was the widely accepted and used expression, because it is simple to apply and use. The wide acceptance of this expression could be realized from the frequent citations of the Mersmann work.8 Apart from the empirical correlations, the interfacial tension can be determined from the crystal growth data using several theoretical models, which include the birth and spread model,10 multiple nucleation model,11 and the Burton-Cabrera-Frank surface diffusion model.12 All of these theoretical expressions either require the information of interfacial tension or, at other times, can be useful * To whom correspondence should be addressed. Phone: 22 508 1678. Fax: +351 22 508 1449. E-mail address: vasanth_vit@ yahoo.com.
in predicting the interfacial tension during the crystal growth process. Irrespective of the importance of this parameter, determination of this parameter still remains difficult, because the required experiments are laborious and time-consuming. A review on several available methods to determine interfacial tension was made by Wu and Nancollas.2 In the present study, an artificial neural network (ANN) was developed and used to predict the interfacial tension between the crystal and the saturated solution. To the best of our knowledge, no attempts have been made so far to determine the interfacial tension at thesolid/liquid interface using artificial neural networks. Artificial neural networks are found to be an excellent option for solving many complex issues. Artificial neural networks are used to correlate the complex relationship between the inputs and outputs, irrespective of the physical meaning or the underlying process of the studied system. ANN consists of an input layer and an output layer interconnected by several nodes. ANN can rapidly process a large amount of information and have excellent generalization capability for noisy or incomplete data. ANN models are flexible and a well-trained ANN can perform well, where empirical modeling is suitable.13 In chemical engineering, ANNs are used to predict the kinetics of solid/liquid sorption systems,14 activity coefficients of aromatic organic compounds,15 and solubility of proteins.16 In the field of crystallization, neural networks were used to model the growth kinetics of sucrose crystals,17 face growth rate of tetragonal lysozyme,18 growth kinetics, and agglomeration rate of ciprofloxacin hydrochloride.19 In the present study, ANN was developed using only the information of concentration of solute in the solid and liquid phases, molecular weight, and density of the crystal at studied temperature, which can be easily determined for any system. Thus, the developed ANN model can be significantly useful to predict interfacial tension for a wide range of crystallization systems. The results of the constructed ANN were compared with the experimentally determined data published in the literature. 2. Neural Network Modeling In the present study, a multiple-layer feedforward network with a input layer (with five neurons), one hidden layer, and
10.1021/ie801666u CCC: $40.75 2009 American Chemical Society Published on Web 03/18/2009
Ind. Eng. Chem. Res., Vol. 48, No. 8, 2009 4161
Figure 1. Structure of the constructed two-layer network and the flow of information within the network.
Figure 2. Training strategy of the constructed feed forward artificial neural network.
one output layer was constructed initially, to predict the interfacial tension at the crystal/solution interface. Multiple-layer feed forward networks allow signals to flow only in one direction (i.e., from input to output) and can approximate any function very well for the given input conditions. Multiple-layer networks can perform linear or nonlinear computations and can approximate any function reasonably well. The detailed structure of the network and the training strategy of the constructed ANN are shown in Figures 1 and 2, respectively. Figure 1 shows the feed forward network with one hidden layer. P1 is the input vector to the hidden layer, and W1 and b1 represent the weight and bias of the hidden layer, respectively. The information from the hidden layer is transferred to the output layer, as shown in Figure 1. The terms P2 represents the output vector and can be determined from the weight W2 and bias b2 of the output layer. In the present study, a tansig function and a purelin function were used as the propagation functions in the hidden layer and in the output layer, respectively. The training strategy of the network is shown in Figure 2. A Levenberg-Marquardt’s training strategy was used. The incorporation of Marquardt’s algorithm into the backpropagation algorithm to train feedforward networks was explained elsewhere.20 As shown in Figure 2, the input vectors and the corresponding output vectors are used to train the network until it approximates the propagation function. The proposed network with a tansigmoid hidden layer and a linear output layer was determined to be capable of approximating the inputs for the corresponding target to predict the interfacial tension at the crystal/solution interface. Thus, the bias and the weights can be obtained from the training procedure, which is based on the experimental data. In the present study, the density of the crystal, the molecular weight of the crystal, the concentration of solute in the liquid phase, the concentration of solute in the solid phase, and the temperature are assumed as input vectors and the corresponding interfacial tension was defined as the output vector. The experimental data were obtained from the works of Mersmann.8 In the present study, experimental data reported for 57 systems were considered. The experimental data were divided into two sets for the training and testing of the ANN. In the present study,
the ANN was trained with inputs from 27 different systems to test the ideal behavior of 28 systems. The data in the training and testing sets can be statistically correlated with σSL, as a function of the logarithmic ratio of solid-phase concentration to liquid-phase concentration with a coefficient of determination of 0.668: σsL ) 7.8685 ln
( )
CS + 14.323 CL
(1)
The training of the ANNs using the Levenberg-Marquardt algorithm is sensitive to the number of neurons in the hidden layer. The greater the number of neurons, the better the performance of the ANN in fitting the data. During the training process, the number of neurons in the hidden layer was changed while optimizing the transfer function for the given input and output vectors. The main problem involved during the optimization of the propagation function during the training process is the overfitting. Overfitting mainly occurs when too many neurons are in the hidden layer. Overfitting can be determined by the large error deviations between the experimental and the ANN predicted crystal growth rate for the new input data. Overfitting refers to exceeding some optimal ANN size that may finally reduce the performance of ANN in predicting the targets. To avoid the problem of overfitting, ANN was trained using the learning dataset with a different number of neurons in the hidden layer, starting from the minimum of one neuron in the hidden layer. Several trials were made by increasing the number of hidden layers to determine the network that best-predicts the targets corresponding to the learning and validation datasets. Another problem involved during the optimization of propagation function during the training process is the overtraining. Overtraining refers to the training time of ANN that will reduce the performance of ANN in predicting the targets. Overtraining will sometimes lead to the poor prediction of targets, because the network will memorize the training examples, but it does not generalize to the new experimental conditions. The problem of overtraining and overfitting can be avoided by regularization. Regularization step modifies the performance of the transfer
4162 Ind. Eng. Chem. Res., Vol. 48, No. 8, 2009
function and reduces the noise avoiding the problem of overtraining and also overfitting. In the present study, to avoid the problems due to the overfitting, a Bayesian regularization technique in combination with the Levenberg-Marquardt’s algorithm was used during the ANN training process. The Bayesian algorithm works best when the network’s input and output are scaled within the range of -1 to +1.21 In the hidden layer, initially three types of transfer functionss namely, the exponential sigmoid tangent, sigmoid, and linear functionsswere tested while training the ANN. The linear function was used at the output layer. A tansigmoid function in the hidden layer and the linear function in the output layer are determined to be excellent in predicting the growth rate of sucrose crystals, irrespective of the initial operating variable conditions. After several trials, the ANN with 5.73 parameters converges for 826 epochs. The number of neurons in the hidden layer for a three-layer network can be calculated using the formula22 NH )
NW - No/p Ni/p + 1 + No/p
(2)
where NW, Ni/p, NH, and No/p represents the total number of parameters, the number of neurons in the input layer, the number of neurons in the hidden layer, and the number of neurons in the output layer, respectively. Thus, from eq 2, a neural network with approximately one neuron in the hidden layer was determined to be successful to model the interfacial tension at the crystal/solution interface under the given input conditions. Bayesian regularization will stop automatically the training process when the algorithm is truly converged. The Bayesian regularization provides a measure of how many weights and biases are effectively used by the network. The Bayesian algorithm effectively uses and decides the number of network parameters and thereby eliminates the guesswork required in selecting optimum network size. Increasing the number of neurons in the hidden layer does not affect the total number of effective parameters. The same number of effective parameters, irrespective of the number of neurons in the hidden layer, confirms that there is no overfitting. The training program of the constructed ANN will be automatically ended when the neural network had truly converged and the network was set ready for the prediction. The network is converged if the sum of the squared errors and the sum of the squared weights are almost constant over several iterations. The normalized targets predicted by ANN were converted back to the original values to obtain the corresponding crystal growth rates. Initially, three feedforward neural networks of different structure were constructed by adjusting the number of inputs, using a Bayesian regularization technique. Initially, the ANN was constructed only with the two inputs, namely, the solid phase and the logarithmic ratio of solid-phase concentration to liquid-phase concentration, followed by the inclusion of the other two parameters (the density of the crystal, and the molecular weight and temperature). The ANN with five input layers trained by Bayesian regularization was determined to better predict the interfacial tension at the crystal/solution interface for the given inputs both in the training and testing processes. This shows that all the inputs are significant to better predict the interfacial tension at the crystal/solution interface. The details of the completely trained ANN used in the present study to predict the interfacial tension at the crystal/solution interface are given in Table 1. Figure 3a shows a plot of the ANN-predicted σSL and σSL from experiments. From Figure 3a, it can be observed that the ANN-predicted σSL was, more or less, in good agreement with
Table 1. Details of the Trained Neural Network Used to Predict the Interfacial Tension at Crystal/Solution Interface type layer 1 (input) layer 3 (output) number of data used for training number of data used for testing (validations set) effective number of parameters number of training epochs function in hidden layer function of output layer
value/comment 5 neurons 1 neuron 28 29 5.73 826 tansigmoid linear
the experimentally determined σSL values. Although in some cases, the predictions are poor, this, nevertheless, is not a serious problem, because the accuracy of the proposed network in predicting σSL values was very useful, to a certain extent. Figure 3a show the predicted σSL values by ANN for 28 systems that are obtained during the training process. Figure 3 also shows the σSL values determined by ANN for 29 new systems, which are kept unaware during the training process. From Figure 3a, it can be observed that the ANN was good enough to predict the σSL values for the new systems, although these data were kept unaware of the network during the training process. This
Figure 3. Plot of experimentally determined σSL values versus σSL values determined (a) by ANN for different systems and (b) using Mersmann expression for different systems.
Ind. Eng. Chem. Res., Vol. 48, No. 8, 2009 4163 Table 2. Relative Error between σSL Determined by ANN and σSL from Experiments and Relative Error between σSL Determined by Mersmann Expression and σSL from Experiments ANN Training Process
ANN Testing Process
crystal/solution σSL,experimental RE (experimental vs RE (experimental vs crystal/solution experimental RE (experimental vs RE (experimental vs system (mJ/m2)a 8 ANN predicted) Mersmann) system (mJ/m2)a ANN predicted) Mersmann) AgBr AgBrO3 AgCl Ag2CrO4 Ag2SO4 BaCO3 BaCrO4 BaMoO4 BaseO4 BaSO4 BaWO4 CaCO3 CaF2 CaMoO4 Ca(OH)2 CaSO4 CaWO4 CdI2 Ch3COOAg KBr KBrO3 KCl KClO3 K2Cr2O7 KH2PO4 KI KIO3 KNO3 a
65 55 92 107 96 115 120 103 88 135 116 76 150 118 66 76 151 19 113 15 44 22 49 33 16 17 44 29
0.6128 0.384618 0.13863 0.037327 0.284729 0.041617 0.03415 0.046311 0.159955 0.061111 0.00369 0.435132 0.31464 0.115339 0.190091 0.009211 0.256331 0.546316 0.443646 0.478533 0.026409 0.187727 0.136694 0.348061 0.932125 0.030824 0.025227 0.059517
1.996923 0.374545 1.206522 0.096262 0.442708 0.006087 0.055833 0.159223 0.05 0.065926 0.106897 0.782895 0.130667 0.117797 0.330303 0.101316 0.172185 0.184211 0.558407 0.046667 0.231818 0.077273 0.395918 0.50303 0.175 0.429412 0.263636 0.368966
K2SO4 Mg(OH)2 MnCO3 NaCl Na2SiF6 Na2S2O3 NH4Br NH4Cl (NH4)2Cr2O7 NH4H2PO4 NH4I NH4NO3 Ni(OH)2 NH4SCN PbCO3 PbC2O4 PbCrO4 Pb(NO3) PbSeO4 PbSO4 SrCO3 SrMoO4 SrSO4 SrWO4 TiBr TiCl TICrO4 TIIO3 TISCN
23 123 104 38 52 16 14 27 53 4 8 27 110 34 125 139 170 3 71 104 116 100 92 62 92 92 108 87 65
0.751043 0.121707 0.0275 0.308947 0.318 0.0335 0.327857 0.090963 0.416717 0.625 0.937 0.506963 0.073091 0.632176 0.079808 0.083453 0.173812 10.258 0.332958 0.02075 0.027466 0.0088 0.105 0.292871 0.195696 0.333326 0.057815 0.027839 0.1468
0.026087 0.380488 0.120192 0.326316 0.188462 0.5875 0.107143 0.318519 0.813208 0.8 0.1 0.777778 1.209091 0.858824 0.2656 0.117266 0.091765 4.033333 0.215493 0.025962 0.131034 0.094 0.093478 0.130645 0.08587 0.048913 0.255556 0.133333 0.035385
Data taken from ref 8.
suggests the robustness of the constructed ANN in predicting the σSL values for any system for given inputs. Although the deviations between experimental and ANN-predicted σSL values were determined to be greater in some systems, during both the training and testing processes, this does not sem to be a serious drawback, because the model offers a simple and convenient way to test its own applicability using a simple procedure. In addition, in the present study, 57 systems were studied in total while constructing and testing the networks. Furthermore, to check the robustness of the constructed ANN, attention was given by validating the constructed network with a greater number of new inputs than that used in the network training process. The ANN shows promising results for inputs used in the training and testing processes, as can be observed from the relative error between the experimental and the ANNpredicted σSL values. In the literature, because of complexities in estimating the interfacial tension using experimental techniques, several semiempirical or empirical expressions have been proposed to predict the interfacial tension from the easily available solubility data. The widely used expression, as observed from the high number of citations in the work by Mersmann, was given by8
()
σSL ) 0.414KT(cSNA)2/3 ln
cS cL
(3)
In addition to the Mersmann expression, several empirical expressions have been proposed by several researchers to predict the interfacial tension readily from the parameters that can be easily determined from simple experiments. For instance, Nielson and Sohnel7 developed a correlation that related interfacial tension with hydration number and solubility, and Sangwal9 reported the relation between surface tension and the
mole fraction of solute molecules. In the present study, only eq 3, which was originally proposed by Mersmann,8 was selected for comparison with the results obtained from ANN, because only this expression has the same parameters, which are used as inputs during the network training process. Figure 3b shows the plot of σSL values predicted using Mersmann expression versus σSL values from experiments. Comparing Figures 3a and 3b, it can be observed that ANN well-predicts the σSL for the studied systems during the training process and also for the new inputs in the testing process. This shows the applicability of the constructed ANN in predicting the interfacial tension only from the knowledge of solubility, density of crystal and molecular weight of crystals, which can be easily determined. Table 2 shows the calculated relative error between the experimental and the ANN predicted σSL values during the training and testing process. Table 2 also shows the relative error between the experimental and the σSL predicted using the empirical expression described in eq 3 for comparison. From Table 2, it can be observed that the constructed ANN better predicts the σSL values for most of the systems, when compared to the empirical expression described in eq 3. For a few systems, eq 3 better-predicts the experimental σSL value than ANN; however, the performance of the ANN was quite sufficient to predict the interfacial tension obtained from the experiments. Both ANN and eq 3 poorly predicted the σSL at the Pb(NO3)/ solution interface. Considering this as an outlier, the average relative error between the experimental σSL and σSL from ANN was determined to be 0.252. While considering the results of σSL during the training process, in most of the cases, ANN better predicts the σSL for the given inputs. In the present study, the ANN was constructed only with very few data and it is always possible to make the performance of ANN better by introducing more data into the inputs. Irrespective of the very low data
4164 Ind. Eng. Chem. Res., Vol. 48, No. 8, 2009
strength used as inputs, the ANN still showed better performance when compared to the empirical expression, which is widely used to predict σSL. The performance of ANN can be improved significantly by the addition of new data to the inputs for different systems. It is always easy and possible to do this whenever new information is available. The objective of the present study is only to show the applicability of the ANN in predicting the interfacial tension using the inputs that can be easily measured (solubility and density of the crystal). The expression of Mersmann was used only for comparison, to show the performance of the ANN with the semiempirical expression in predicting the σSL. Remember that ANNs are phenomenological and do not inherently produce a mechanistic understanding of the physical phenomena.22 In this study, the ANN was determined to be a successful tool to predict the interfacial tension at the crystal/solution interface. Despite the modern technologies, precise estimation of the parameter σSL still remains a difficult task. Although this parameter can be obtained from crystal growth experiments using the nucleation theory, it requires a great amount of experimental investigation and time. The proposed network predicts the value of σSL, with a relative error in the range of 0.009-0.932 to 0.027-0.75 during the training and testing processes, respectively. This shows that the constructed ANN was successful trained to predict the value of σSL for the inputs (solubility and density), which can be determined easily from the experiments. Proposing a purely theoretical model to predict σSL directly only from the solubility data is very difficult task. In the present study, the trained ANN predicted this parameter with considerable accuracy. The proposed ANN can be used to predict the interfacial tension at the crystal/liquid interface. The constructed m.file, which may be useful to predict σSL for other systems and for the systems reported, is available for research purposes and can be obtained from the Supporting Information. Supporting Information Available: Constructed m.file data, which may be useful to predict the σSL values for other systems and for the systems reported in this work. This material is available free of charge via the Internet at http://pubs.acs.org. Literature Cited (1) Rousset, Ph.; Sellappan, P.; Daoud, P. Effect of emulsifiers on surface properties of sucrose by inverse gas chromatography. J. Chromatogr. A 2002, 969 (1-2), 97. (2) Wu, W.; Nancollas, G. H. Determination of interfacial tension from crystallization and dissolution data: A comparison with other methods. AdV. Colloid Interface 1999, 79 (2-3), 229.
(3) Mullin, J. W. Crystallization; Butterworth-Heinemann: Woburn, MA, 2001. (4) Digilov, R. M. Solid-liquid interfacial tension in metals: correlation with the melting point. Phys. B 2004, 352 (1-4), 53. (5) So¨hnel, O. Electrolyte crystal-aqueous solution interfacial tensions from crystallization data. J. Cryst. Growth 1982, 57 (1), 101. (6) Bennema, P.; So¨hnel, O. Interfacial surface tension for crystallization and precipitation from aqueous solutions. J. Cryst. Growth 1990, 102 (3), 547. (7) Nielsen, A. E.; So¨hnel, O. Interfacial tensions electrolyte crystalaqueous solution, from nucleation data. J. Cryst. Growth 1971, 11 (3), 233. (8) Mersmann, A. Calculation of interfacial tensions. J. Cryst. Growth 1990, 102 (4), 841. (9) Sangwal, K. On the estimation of surface entropy factor, interfacial tension, dissolution enthalpy and metastable zone-width for substances crystallizing from solution. J. Cryst. Growth 1989, 97 (2), 393. (10) Burton, W. K.; Cabrera, N.; Frank, F. C. The growth of crystals and the equilibrium structure of their surfaces. Philos. Trans. R. Soc., A 1951, 243 (866), 299. (11) Sangwal, K. Effects of impurities on crystal growth processes. Prog. Cryst. Growth. Chacter. 1996, 32 (1-3), 3. (12) Kumar, C. A new look at BCF surface diffusion model. J. Cryst. Growth 1980, 48 (3), 489. (13) Bryjak, J.; Ciesielski, K.; Zbicin´ski, I. Modelling of glucoamylase thermal inactivation in the presence of starch by artificial neural network. J. Biotechnol. 2004, 114 (1-2), 177. (14) Kumar, K. V.; Porkodi, K.; Rondon, R. L. A.; Rocha, F. Neural network modeling and simulation of the solid/liquid activated carbon adsorption process. Ind. Eng. Chem. Res. 2008, 47 (2), 486. (15) Chow, H.; Chen, H.; Ng, T.; Myrdal, P.; Yalkowsky, S. H. Using backpropagation networks for the estimation of aqueous activity coefficients of aromatic organic compounds. J. Chem. Inf. Comput. Sci. 1995, 35 (4), 723. (16) Naik, A. D.; Bhagwat, S. S. Optimization of an Artificial Neural Network for Modeling Protein Solubility. J. Chem. Eng. Data 2005, 50 (2), 460. (17) Kumar, K. V.; Martins, P.; Rocha, F. Modelling of the batch sucrose crystallization kinetics using artificial neural networks: comparison with conventional regression analysis. Ind. Eng. Chem. Res. 2008, 47 (14), 4917. (18) Noever, D.; Pusey, M. L.; Forsythe, E. L.; Baskaran, S. Artificial neural network prediction of tetragonal lysozyme face growth rates. J. Cryst. Growth 1996, 167 (1-2), 221. (19) Yang, M.; Wei, H. Application of a neural network for the prediction of crystallization kinetics. Ind. Eng. Chem. Res. 2006, 45 (1), 70. (20) Hagan, M. T.; Menhaj, M. B. Training feedforward networks with the Marquardt algorithm. IEEE Trans. Neural Networks 1994, 5 (6), 989. (21) Demuth, H.; Beale, M. In Neural Network Toolbox User’s Guide; Mathworks: Natick, MA, 2000: pp 5-1-5-75. (22) Sundaram, N. Training neural networks for pressure swing adsorption processes. Ind. Eng. Chem. Res. 1999, 38 (11), 4449.
ReceiVed for reView November 2, 2008 ReVised manuscript receiVed February 17, 2009 Accepted March 4, 2009 IE801666U
