QSTR Modeling for Qualitative and Quantitative Toxicity Predictions of

Aug 28, 2014 - Therefore, the regulatory agencies have been stressing for toxicity evaluation of the ... The artificial intelligence (AI) based models...
0 downloads 0 Views 827KB Size
Download PDF
Article pubs.acs.org/crt

QSTR Modeling for Qualitative and Quantitative Toxicity Predictions of Diverse Chemical Pesticides in Honey Bee for Regulatory Purposes Kunwar P. Singh,*,†,‡ Shikha Gupta,†,‡ Nikita Basant,§ and Dinesh Mohan∥ †

Academy of Scientific and Innovative Research, Anusandhan Bhawan, Rafi Marg, New Delhi-110 001, India Environmental Chemistry Division, CSIR-Indian Institute of Toxicology Research, Post Box 80, Mahatma Gandhi Marg, Lucknow-226 001, India § Kanban Systems Pvt. Ltd., Laxmi Nagar, Delhi-110092, India ∥ School of Environmental Sciences, Jawaharlal Nehru University, New Delhi-110067, India ‡

S Supporting Information *

ABSTRACT: Pesticides are designed toxic chemicals for specific purposes and can harm nontarget species as well. The honey bee is considered a nontarget test species for toxicity evaluation of chemicals. Global QSTR (quantitative structure−toxicity relationship) models were established for qualitative and quantitative toxicity prediction of pesticides in honey bee (Apis mellifera) based on the experimental toxicity data of 237 structurally diverse pesticides. Structural diversity of the chemical pesticides and nonlinear dependence in the toxicity data were evaluated using the Tanimoto similarity index and Brock−Dechert−Scheinkman statistics. Probabilistic neural network (PNN) and generalized regression neural network (GRNN) QSTR models were constructed for classification (two and four categories) and function optimization problems using the toxicity end point in honey bees. The predictive power of the QSTR models was tested through rigorous validation performed using the internal and external procedures employing a wide series of statistical checks. In complete data, the PNN-QSTR model rendered a classification accuracy of 96.62% (two-category) and 95.57% (fourcategory), while the GRNN-QSTR model yielded a correlation (R2) of 0.841 between the measured and predicted toxicity values with a mean squared error (MSE) of 0.22. The results suggest the appropriateness of the developed QSTR models for reliably predicting qualitative and quantitative toxicities of pesticides in honey bee. Both the PNN and GRNN based QSTR models constructed here can be useful tools in predicting the qualitative and quantitative toxicities of the new chemical pesticides for regulatory purposes.

1. INTRODUCTION

regulatory agencies have been stressing for toxicity evaluation of the existing pesticides as well as new chemicals coming up as their substitutes. Accordingly, standard experimental protocols have been established by the industry and government agencies to test chemicals for their toxic potential. However, a computational approach in chemical toxicology continues to be an attractive viable approach to reduce the amount of efforts and cost of experimental toxicity assessment4 and overall its ability to predict the potential property of a new chemical before its synthesis and biological testing. The recently approved European Union (EU) regulation “Registration, Evaluation and Authorization of Chemicals (REACH)”5 advocates the use of non-animal testing methods and in particular quantitative structure−toxicity/activity relationship (QSTR/QSAR) approaches in order to decrease the number and cost of animal

Pesticides constitute an important class of chemicals as these are designed toxic molecules for specific purposes with widespread usage in crop protection and disease vector control programs. India being the second largest manufacturer of pesticides uses an average amount of 381 g/ha (crop area) of technical grade pesticides against the global average of 500 g/ ha.1 Because of the inherent properties of these chemicals, a major fraction of their applied dose remains present as residues on the target vegetation and field soils. Subsequently, considerably higher concentrations of the pesticide residues are reported in vegetation, crops, and other edible products2 and cause exposure to humans and animals. Long-term contact with these chemicals can harm human life and can disturb the function of different organs in the body, including the nervous, endocrine, immune, reproductive, renal and cardiovascular, and respiratory systems.3 Moreover, exposure to these chemicals is reported to be the reason for the extinction of several avian and other animal species during past decades. Therefore, the © 2014 American Chemical Society

Received: March 19, 2014 Published: August 28, 2014 1504

dx.doi.org/10.1021/tx500100m | Chem. Res. Toxicol. 2014, 27, 1504−1515

Chemical Research in Toxicology

Article

have a network that can provide mapping for one set of sample points to another and if the mapping is one to one, can easily generate inverse mapping from the same sample points;20 are methods that can be learned quickly and produce reproducible outputs without any risk of a local minimum of error surface;21 and have successfully been applied in toxicity prediction of the organic chemicals.16−19 However, the main disadvantage of PNN/GRNN relative to other techniques is that these require substantial computation to evaluate new points.20 In this study, the PNN and GRNN based QSTR models were established for qualitative (toxicity classes) and quantitative (pLD50) toxicity prediction of structurally diverse chemical pesticides in HB22 using simple molecular descriptors derived from chemical structures. The predictive and generalization abilities of the proposed QSTR models constructed here were evaluated using several statistical criteria. The external predictive power of the QSTR model was evaluated using OECD recommended external validation tests. Performance of the QSTR models obtained with the optimal descriptors and considering the larger data set here for predicting the toxicity of pesticides in HB has been superior to those reported earlier.

testing. Consequently, the development and use of QSTR/ QSAR models for the prediction of the ecotoxicity of chemicals have increased dramatically over the past decades.6,7 However, most critical limitations of many QSAR studies are their low external predictive power when applied to predict the toxicity of new chemicals not included in the model building phase, which could be due to the incorrect usage or lack of external validation during the modeling process. A poor performance of the QSAR model in both the training and test phases may be due to the selection of inappropriate modeling methods or irrelevant descriptors. Also, the experimental toxicity data generally have nonlinear structure, and linear methods fail to capture the nonlinear dependence. Moreover, the majority of the QSARs in the past have been developed for addressing the problem of the aquatic toxicity of various groups of chemicals in aquatic test species, and on the whole, toxicological predictions in terrestrial species are ignored. Since, pesticides are mainly applied to crops through aerial spray methods, their residual effects are more significant and realistically need to be screened in terrestrial species. The honey bee (Apis mellifera) (HB) is probably among the most exposed species to the residues of pesticides applied to the crops. HB provides important services to the agricultural ecosystems as these are efficient pollinators in addition to the production of honey, a higher commercial value product. A recent decline in population of HB in various countries has been linked with pesticide toxicity.8−11 HB is an EPA recommended (PMRA DIR 2001-02) nontarget test species for terrestrial toxicity assessment of chemicals. Despite the importance of HB and the need for terrestrial toxicological assessment of pesticides, only a few studies have reported QSARs on the subject. Vighi et al.12 proposed a QSTR model for estimating the acute toxicity of organophosphorus pesticides to Apis mellifera. Devillers et al.13 proposed a neural network based QSAR model for predicting the toxicity of pesticides to Apis mellifera. Cheng et al.14 developed both the classification ̈ and regression QSARs for pesticide toxicity to HB using Naive base, decision tree, k-nearest neighbors, random forest, and support vector machine approaches. Toropov and Benfenati10 proposed a QSAR model for predicting the toxicity of pesticides in HB using the SMILES (simplified molecular input line entry system) based descriptors, which help in enumerating the toxicity of respective fragments. However, these studies used a limited data set, or the predictive performance of the proposed QSTRs was not satisfactory, thus limiting their scope. Therefore, there is a need for developing robust QSTR models both for qualitative and quantitative predictions of the toxicity of pesticides in HB. The artificial intelligence (AI) based models, in general, are capable of capturing the complex nonlinear relationships between the relevant descriptors (or properties) and observed responses.15 Among these, artificial neural networks (ANNs) are universal estimators and have emerged as unbiased tools of prediction of response using a set of independent estimators. Probability density function (PDF) based neural networks such as probabilistic neural networks (PNNs) and generalized regression neural networks (GRNNs) have successfully been used in various classification and regression QSTR modeling studies.16−19 The advantages of these methods relative to other nonlinear techniques are that they operate completely in parallel without a need for feedback from the individual neurons to the inputs; tolerate erroneous samples; estimate observations that are bound by the minimum and maximum;

2. MATERIALS AND METHODS 2.1. Data Set. US EPA data on pesticide toxicity in honey bee (Apis mellifera)22 was considered for the QSTR analysis in this study. The chemically heterogeneous data set comprises a total of 303 registered pesticides (insecticides, herbicides, and fungicides) representing different groups and is known for a wide spectrum of mechanisms of toxic actions. For these pesticides, values of 48-h exposure based median lethal dose (LD50, μg/bee) for honey bee (Apis mellifera) are reported. In order to get a high quality data set, a rigorous selection process was applied here. All of the mixtures, salts, and the compounds which have too little structural information were removed from the original data set. After this process, 66 compounds were excluded, and 237 pesticides were retained for QSTR modeling. The toxicity values of these compounds ranged between 0.01 μg/bee and 1401.46 μg/bee (Table S1, Supporting Information). For the development of the GRNN-QSTR model, LD50 values (μg/bee) were converted to molar basis, and then the log transformed data (−log LD50, pLD50) were used as the response variable. For a robust QSTR model, it is required that the compounds considered in the model building phase should be structurally diverse. The structural diversity of the considered data set was assessed by the calculation of the average Tanimoto similarity index (TSI) based on the molecular descriptors. TSI, an appropriate distance metric for topology-based chemical similarity studies are calculated using Tanimoto similarity between the fingerprint of a chemical and a consensus fingerprint, which is a 1024 bit fingerprint (Toxmatch, Ideaconsult Ltd.). The fingerprint generation is based on the fingerprint implementation of the open source chemoinformatics library.23 For a given molecule, all possible paths for a predefined length are generated, the path is submitted to a hash function, which uses it as a seed to a pseudorandom generator, and the hash function puts out a set of bits, which is added to the fingerprint. For a molecule, TSI is calculated as TSIAB = 2ZAB[ZAA + ZBB − ZAB]−1, where Z is the similarity matrix, and A and B are the two molecules being compared. The TSI ranges from 0 (no similarity) to 1 (pairwise similarity). A good cutoff for biologically similar molecules is 0.8. Smaller TSI ( 100 μg/bee) and among four toxicity classes generated according to the intervals established by the USEPA criteria,22 LD50 ≤ 2 μg/bee for class 1 (highly toxic), 2 μg/ bee < LD50 ≤ 11 μg/bee for class 2 (moderately toxic), 11 μg/bee < LD50 ≤ 100 μg/bee for class 3 (slightly toxic), and LD50 > 100 μg/bee for class 4 (non toxic), was performed. For regression modeling, the end point toxicity (LD50) values were expressed as pLD50 (mmol/ bee). 2.3. Selection of Modeling Approach for QSTR Model Development. In the present study, the BDS statistics exceeded the significance level (p < 0.01), thus suggesting for severe nonlinear data structure, it was appropriate to select a nonlinear modeling approach for establishing the QSTR models for predicting the toxicities of pesticides in HB. Accordingly, we selected the nonlinear PNN and GRNN modeling approaches for the development of QSTR models for qualitative (toxicity classes) and quantitative (end point toxicity values) predictions of toxicities of structurally diverse chemical

Figure 1. Histogram of Tanimoto distance values of the pesticides in the complete data set (HB). Linear modeling methods fail to capture the nonlinearities in data, resulting in poor predictive performance of the constructed models. Nonlinearity in the experimental toxicity data was tested using the Brock−Dechert−Scheinkman (BDS) statistics.25 It tests the null hypothesis of independent and identically distributed (I.I.D.) data against an unspecified alternative. The BDS statistics is defined25 as BDSε,m = N1/2[Cε,m − (Cε,m)m]/σε,m, where σε,m is the standard deviation of Cε,m. If the computed BDS statistics exceed the critical value at the conventional level, the null hypothesis of linearity is rejected, which reveals the presence of nonlinear dependence in the data.26 2.2. Molecular Descriptors, Feature Selection, and Data Processing. Molecular descriptors are simple mathematical representations of a molecule and are used to encode their significant features. In this study, 174 molecular descriptors (topological, electronic, geometrical, and constitutional) were calculated for each pesticide using the Chemistry Development Kit (CDK v 1.0.3).27 These descriptors were calculated by 2D structures of the molecules, which were taken in the form of SMILES (simplified molecular input line entry system). SMILES for the pesticides were obtained using Chemspider.28 SMILES has the advantage of taking into account some important molecular features, such as the presence of cycles, cis-/transisomerism, and the presence of sp2 and sp3 atoms.29 For selection of relevant features, model-fitting approaches were considered. The descriptors exhibiting a constant or near constant value and those with low variation were excluded from the original pool. Further, nonlinear modeling was performed using the PNN and GRNN approaches. For optimal values of the kernel function

Table 1. Selected Descriptors in Classification and Regression QSTRs with Their Descriptions descriptors

class

TopoPSA WTPT-4 WTPT-5 BCUTw-1h Wlambda2.unity WV.unity chi1vC IonzPot PNSA-2 WPSA-1 RPCS

topological topological topological topological topological topological topological electronic electronic electronic electronic

THSA RPSA

electronic electronic

MOMI-XZ XLogP nRotB nAtomLC nAromBond

geometrical constitutional constitutional constitutional constitutional

explanation topological polar surface area based on fragment contributions weighted path descriptors of the sum of path lengths starting from oxygen’s weighted path descriptors of the sum of path lengths starting from nitrogen’s the number of the lowest eigenvalue of the highest atom weighted BCUTS holistic descriptors holistic descriptors carbon valence connectivity index (order 1) ionization potential partial negative surface area × total negative charge on the molecule partial positive surface area: sum of the surface area on positive parts of the molecule × total molecular surface area/1000 relative positive charge surface area: most positive surface area × relative positive charge; most positive charge/total positive charge sum of solvent accessible surface areas of atoms with an absolute value of partial charges less than 0.2 sum of solvent accessible surface areas of atoms with absolute value of partial charges greater than or equal to 0.2/total molecular surface area moment of inertia along X/Z axis logarithmic form of octanol−water partition coefficient based on the atom type method number of rotatable bonds number of atoms in the largest chain number of aromatic bonds 1506

dx.doi.org/10.1021/tx500100m | Chem. Res. Toxicol. 2014, 27, 1504−1515

Chemical Research in Toxicology

Article

Table 2. Basic Statistics of the Selected Descriptors in Classification and Regression QSTRsa

a

descriptors

model

min

max

mean

SD

CoV

TopoPSA WTPT-4 WTPT-5 BCUTw-1h Wlambda2.unity WV.unity chi1vC IonzPot PNSA-2 WPSA-1 RPCS THSA RPSA MOMI-XZ XLogP nRotB nAtomLC nAromBond -log LD50 (mmol/bee)

R, C4 C4 C4 R, C4 C2 C4 R C4 C4 C4 C4, C2 R R R R, C2 C2 C4, C2 R R

7.76 0.00 0.00 14.01 0.28 2.59 0.00 −1.00 −1638.62 17.50 0.00 25.95 0.00 1.60 −1.18 0.00 0.00 0.00 2.27

208.74 23.03 18.42 78.92 8.37 463.42 11.83 10.05 −60.82 703.31 18.84 876.13 0.90 38.86 7.81 16.00 44.00 18.00 7.46

73.91 7.51 6.67 29.67 2.84 101.34 3.81 7.46 −424.64 258.67 1.94 534.84 0.16 5.20 3.16 7.81 12.52 7.86 4.00

36.40 4.67 4.90 10.89 1.22 73.89 1.99 2.81 267.83 116.77 3.02 126.55 0.10 3.88 1.80 3.56 8.15 4.87 1.08

49.25 62.15 73.53 36.71 42.98 72.91 52.18 37.70 −63.07 45.14 155.63 23.66 65.01 74.57 56.87 45.61 65.12 62.01 26.91

C2, two-category classification; C4, four-category classification; R, regression; SD, standard deviation; CoV, coefficient of variation. to a later node are the coordinates of the corresponding sample. The node computes the distance d(s,x) from the test vector x to the training samples and put out the value of the kernel function. The outcome of each of the layer one cells is added separately for the different classes by the connections to the output cells with weight one. 2.3.2. GRNN-QSTR Model. GRNN uses nonlinear Gaussian kernel regression and estimates any arbitrary function between the input and output vectors, drawing the function estimate directly from the training data and providing the optimal estimation of continuous variables implementing the statistical concepts of conditional probability.33 It is a four-layered NN (Figure 2b) consisting of an input layer, a pattern layer, a summation layer, and an output layer, and does not require any iterative training procedure to converge to the desired solution. The input layer provides input vector x to pattern layer, which consists of neurons for each training datum or for each cluster center. In this layer, the weighted squared Euclidean distance is calculated according to D2i = (x − xi)T(x − xi), where xi refers to the ith training vector. Any new input applied to the network is first subtracted from the pattern layer neuron values, then according to the distance function, either squares or absolute values of subtractions are summed and applied to activation function. Results are transferred to the summation layer. Neurons in the summation layer add a dot product of the pattern layer outputs and weights. At the output layer, multiplication of pattern layer outputs and training data output Y values (Yf(x)K) are divided by weighted outputs of the pattern layer ( f(x)K) to estimate the desired Y. 2.4. Model Architecture Optimization. The pesticide toxicity data were split into training and test sets, and the two- and fourcategory classification (PNN) and regression (GRNN) QSTR models were developed using the training set while keeping the test data for external validation of the constructed models. The optimal architectures and model parameters of the QSTR models (PNN and GRNN) and number of relevant descriptors were determined following the V-fold cross-validation procedure. Optimal models were selected on the basis of the classification accuracy (classification) and mean squared error (MSE) in the training and validation data.34 The advantage of this method is that it performs reliable and unbiased testing on the data set.35 Since selection of the relevant descriptors is of great importance in predictive QSTRs, the developed classification and regression models were further validated using the Y-randomization test. Y-randomization has been frequently used to determine the possibility of the chance of

pesticides in honey bee using a set of simple molecular descriptors. PNN and GRNN approaches have successfully been applied in computational toxicology earlier.18,19 A brief account of these approaches is provided here: 2.3.1. PNN-QSTR Model. PNN, a classification method,20 estimates the probability density function (PDF) of the features of each class from the available training data. Estimated PDFs are then used in a Bayes decision rule to perform the classification.32 PNN uses a nonparametric technique (Parzen window) to construct the classdependent PDF for each classification category required by the Bayes’ theory. PNN architecture (Figure 2a) consists of a node in layer one for each of the N training samples. The weights leading from the input

Figure 2. Architecture of the (a) PNN-QSTR and (b) GRNN-QSTR models. 1507

dx.doi.org/10.1021/tx500100m | Chem. Res. Toxicol. 2014, 27, 1504−1515

Chemical Research in Toxicology

Article

correlation during the descriptor selection procedure.30 In Yrandomization, the dependent variable (category and pLD50) vectors were randomly shuffled, and new models were built using the original independent variables.36 The procedure was repeated a number of times, and CV statistics were computed. The performance (misclassification rate and R2) of the new models was compared with the original models. If the new models have a higher misclassification rate (classification) and lower R2 values (regression) for several trials, then the given model is thought to be robust. Thus, Y-randomization is useful to avoid any chance correlation between the dependent variable vector and independent variables.36 2.5. External Validation of Developed QSTR Models. In case of the predictive QSTR models, the validation step using the external data set provides information about the predictive ability of the trained model for the unknown data.19 Benigni et al.37 pointed out that the prediction reliability should be checked by means of an external test set with new chemicals not used in model building. For external validation, a separate validation (test) subset of the data was used, which was kept out during the training process.34 The predictive power of the regression QSTR model (GRNN) constructed here for the external set was evaluated using various OECD38 recommended validation criteria parameters proposed in QSAR literature.39 Lin40 proposed the concordance correlation coefficient (CCC) for external validation. This coefficient measures both precision and accuracy; consequently, any divergence of the regression line from the concordance line gives as a result a value of CCC smaller than 1. In CCC, no training set information is involved, so it can be considered a true external validation measure independent of the sampled chemical space. Shi et al.41 proposed criteria of Q2F1 which uses the average of the training data instead of that of the prediction set. Schuurmann et al.42 proposed alternative criterion of Q2F2 for external validation. It differs from the earlier one because the average value at the denominator is calculated using the prediction set instead of the training one. However, Q2F2 takes no account of the distance from the average of the training values. Consonni et al.43 proposed a new external validation measure Q2F3, comparing and highlighting differn ences with Q2F2 and can be calculated as Q2F3 = 1 − [∑i=1ext(ŷi − yi)2]/ next/[∑ni=1Tr (yi − yT̅ r)2]/nTr, where nTr is the number of compounds in the training set, next is the number of compounds in external (test) set, yi and ŷi are the observed and model calculated value of the dependent variable, respectively, in the external set, and yt̅ r is the mean value of the dependent variable in the training set. In Q2F3, the denominator is calculated on the training set, and both numerator and denominator are divided by the number of corresponding elements. It may be noted that the results obtained by Q2F3 are independent of the prediction set distribution and sample size.43 Recently, Roy et al.44 proposed a novel metric r2m as an additional validation parameter; r2m = r2(1 − (r2 − r2o )1/2). This metric is calculated based on the correlations between the observed and predicted values with (r2) and without (r2o ) the intercept for the least-squares lines and does not consider the differences between individual responses and the training set mean and thus avoids overestimation of the quality of prediction due to a wide response range (Y-range). Additionally the criteria, R2 − R2o /R2 < 0.1 and 0.85 ≤ k ≤ 1.15 were also applied to the test set to further assess the predictive power of the proposed QSTR model.45 Here, R2 is the squared correlation coefficient between observed and predicted values for the test set; R2o and k are the correlation coefficient and the slopes of the linear regression between the observed and predicted values when the intercept was set to zero. The performance of the proposed QSTR model in predicting the toxicity of pesticides was also assessed by calculating R2 and the root mean squared error (RMSE) of the training, test, and complete data arrays. Further, the performance of the classification (PNN) models for the two- and four-toxicity category classification of compounds was assessed in terms of the sensitivity, specificity, and accuracy of prediction.34 Sensitivity is of the utmost importance to track false negatives whose number should be kept low to avoid the occurrence of toxic compounds wrongly predicted as not hazardous, whereas specificity is concerned with the number of negative compounds correctly predicted, and its value decreases with the occurrence of false

positives. Accuracy, also termed concordance, measures the correctness of the prediction. Its value is obtained by dividing the number of correct predictions by the total number of compounds. 2.6. Applicability Domain of the QSTR Models. Any given QSTR model should make reliable predictions within its applicability domain (AD) for untested compounds. In this study, the chemical domain of the pesticides was investigated using two different approaches. The first approach is based on the ranges of individual descriptors used for model building. According to this method, a compound with descriptor values within the range of those of the training set compounds is considered as being inside the AD of the model.46 The second method was based on the leverage approach.47 The leverage value, hi for each ith chemical is calculated from the descriptor (i × j) matrix (X) as hi = xTi (XTX)−1xi, where xi is a raw vector of molecular descriptors for a particular ith compound. To visualize the AD of the QSTR model (GRNN), the standardized residuals were plotted against leverages (Williams plot). The obtained Williams plot was used to detect both the response outliers and the structurally influential chemicals in the model. In this plot, the horizontal and vertical lines delineate the limit of acceptable values, the former for Y-outliers (i.e., compounds with standardized residuals greater than 3 SD unit) and the latter for X-outliers, respectively. The limit of X-outliers is determined by their warning hat value (h*) calculated47 as h* = 3(p + 1)/n, where p is the number of variables used in the model, and n is the number of training compounds. The value of hi greater than the critical h* value indicates that the structure of the compound substantially differs from those used for the calibration. Therefore, the compound is located outside the optimum prediction space. Although, this approach for identifying the potential outliers has its limitation due to nonavailability of the toxicity data for the new test molecules, it has been followed here in view of its wide use in QSAR literature.

3. RESULTS AND DISCUSSION Basic statistics of the selected molecular descriptors are provided in Table 2. Values of the standard deviation and coefficient of variation (CoV) suggest that the selected descriptors exhibited high variability. The CoV values of the descriptors ranged between −63.07% (PNSA-2) and 155.63% (RPCS). The electronic descriptors exhibited the highest variability (−63.07%−155.63%) followed by topological (36.71%−73.53%), geometrical (74.57%), and constitutional (45.61%−65.12%) ones. 3.1. Qualitative QSTR Modeling. A qualitative QSTR model was constructed to categorize the pesticides among toxic and nontoxic (two-categories) as well as among highly toxic, moderate toxic, slightly toxic, and nontoxic (four-categories) categories. Accordingly, PNN based QSTR models were established for two- and four-category classifications of the considered pesticides using the set of selected molecular descriptors (Table 2). Optimal architecture and the model parameters were determined through 5-fold CV, whereas for external validation, a subset of test data was used. The classification accuracies obtained in CV of two- and fourcategory QSTRs ranged between 70.21% and 77.08% and 74.47% and 85.94%, respectively. The results indicate that the toxicity prediction accuracies of both the QSTR models are comparable in two- and four-category classifications. The results have also shown no obvious overfitting of the data. The Y-randomization tests were performed both for the twoand four-category classification of pesticides using the 5-fold CV procedure. Average values of the misclassification rate in two- and four-category QSTRs were 27.51% and 43.67%, respectively, which are significantly higher (3.38% and 8.86%) than those of the respective original classification QSTRs. This 1508

dx.doi.org/10.1021/tx500100m | Chem. Res. Toxicol. 2014, 27, 1504−1515

Chemical Research in Toxicology

Article

suggests that the original classification QSTR models are relevant and unlikely to arise as a result of chance of correlation. 3.1.1. Qualitative QSTR (Two-Category). The selected fourlayered optimal PNN-QSTR model has 5 neurons in the input layer, 175 neurons in the pattern layer, 2 neurons in the decision layer, and 1 neuron in the output layer. The value of the spread (σ) parameter of the reciprocal function was optimized. Here, separate σ values were considered for each of the input variables, and a search for each was made in the range of 0.001−10. Selection of σ values for each variable provided a relatively better model as compared to the single model σ value. The optimal values of σ for the considered input variables ranged between 0.025 and 0.065. The optimal PNN-QSTR model was applied to the test and complete data arrays. The discriminating descriptors for two-category classification of pesticides in the QSTR model were determined in view of their importance in the corresponding model. The contribution of the selected descriptors ranged between 63.82% and 100% (Figure 3a). It is evident that the contribution of constitutional

The performance parameters of the constructed PNN-QSTR model for the training, test, and complete data are presented in Table 3. The model rendered classification accuracies of 100% Table 3. Performance Results for the PNN-QSTR Classification Models subsets

total cases

training test complete

175 62 237

training test complete

178 59 237

sensitivity (%)

specificity (%)

Two-Category Classification 100.00 100.00 85.96 100.00 95.53 100.00 Four-Category Classification 97.51 98.20 85.44 91.63 94.29 96.56

accuracy (%) 100.00 87.10 96.62 97.47 89.83 95.57

in training and 87.10% in test data. The overall sensitivity, specificity, and accuracy in complete data were 95.53%, 100%, and 96.62%, respectively. The results suggest that the performance of the proposed QSTR model was reasonably good in discriminating the toxic and nontoxic pesticides according to the criteria laid down.22 Cheng et al.14 proposed ̈ Bayes, k-NN, binary classification QSTRs based on Naive random forest, decision tree, and support vector machine methods for pesticides (N = 195) using 166 MACCS structural keys and 307 fingerprints (FP4) as estimators and reported classification accuracies in the range 67.5% and 95%. Although, accuracies achieved using MACCS estimators were within the acceptable range, these are complex descriptors. 3.1.2. Qualitative QSTR (Four-Category). The four-category optimal PNN-QSTR model has 10 neurons in the input layer, 178 neurons in the pattern layer, 4 neurons in the decision layer, and 1 neuron in the output layer. The value of the spread (σ) parameter of the Gaussian function was optimized considering separate σ values for each of the input variables, and a search for each was made in the range of 0.001−10. The optimal σ values ranged between 0.458 and 1.913. The optimal PNN-QSTR model was applied to the test and complete data arrays. The contribution of the selected descriptors ranged between 6.46% and 100% (Figure 3b). It is evident that the contribution of the electronic and topological descriptors ranged between 6.46% and 100% and 8.51% and 23.72%, respectively, whereas the contribution of the lone constitutional descriptor was 57.18%. IonzPot exhibiting the highest contribution (100%) describes the energy needed for the ionization of a chemical. Compounds that ionize in aqueous medium do not bioaccumulate, as they cannot cross biological membranes. Furthermore, it may have a relationship with the polarization effect of chemicals and consequently with their reactivity toward the numerous nucleophilic binding sites present in living organisms.51 The mean sensitivity, specificity, and accuracy values yielded by the constructed QSTR model for the training, test, and complete data are summarized in Table 3. It is evident that the model in complete data yielded sensitivity, specificity, and accuracy values of 94.29%, 96.56%, and 95.57%, respectively. However, the model yielded a classification accuracy of 89.83% in the test phase. Moreover, the model rendered the classification accuracy of 94.92% for highly toxic, 98.31% for moderate toxic, 79.66% for slightly toxic, and 86.44% for nontoxic chemicals in the test phase. A relatively lower accuracy in the case of slightly toxic compounds may be due to the larger number of chemicals in this category.

Figure 3. Plot of the contribution of input variables in (a) twocategory classification PNN-QSTR and (b) four-category classification PNN-QSTR models.

descriptors ranged between 84.25% and 100%, whereas the lone topological and electronic descriptors contributed 87.73% and 63.82%, respectively. Constitutional descriptors usually represent counts of different types of atoms or bonds. While very simplistic, they play a useful role in predictive modeling.48 Among the constitutional descriptors, nRotB exhibited the highest (100%) contribution. It represents the conformational rigidity of the molecules, which is important for biological activity.49 The importance of the independent variables in the model was determined using the difference in misclassification rate (MR) calculated using the actual data values of all predictors and those computed through randomly rearranged values of each predictor.50 1509

dx.doi.org/10.1021/tx500100m | Chem. Res. Toxicol. 2014, 27, 1504−1515

Chemical Research in Toxicology

Article

Figure 4. Plot of the contribution of input variables in the GRNN-QSTR regression model.

Fjodorova et al.52 suggested that the predictive QSTR model for regulatory purposes should be connected with high sensitivity. A low sensitivity value indicates the low ability of a model to recognize the toxicity of diverse compounds, a high specificity value indicates the high ability of the model to recognize the false positive compounds, and it can save on experimental costs.24 Accuracy, however, represents the total number of active and inactive compounds correctly predicted among the total number of the tested compounds. The performance parameters (sensitivity, specificity, and accuracy) for both the two- and four-category classification QSTRs established for pesticide toxicity suggest that both of these are fully capable of discriminating the chemicals in different toxicity classes. An in-depth investigation of the results revealed that the proposed two- and four-category classification models in complete data misclassified 8 and 21 compounds, respectively. In two-category classification, all of the misclassified compounds belong to the second category (nontoxic). These compounds are mainly fungicides (4), herbicides (3), and insecticides (1) and belong to the groups of the neutral organics (5), ester (1), benzyl alcohol (1), and propargyl ether (1). In four-category classification, a majority of the misclassified compounds (18) belong to the third (slightly toxic) and fourth (nontoxic) categories. The misclassified compounds are mainly herbicides (9), fungicides (5), insecticides (4), and plant growth regulators (3). The results suggest that both the two- and four-category classification models misclassified mainly the nontoxic compounds and correctly captured the toxic ones. This may be due to the fact that the misclassified compounds were not appropriately represented by the set of selected descriptors in the classification models. 3.2. Quantitative QSTR Modeling. A quantitative QSTR model was constructed for predicting the toxicity (pLD50) of diverse pesticides using the GRNN approach. Optimal architecture and the model parameters were determined using the 5-fold CV, whereas for external validation, a subset of data were used. A criterion of minimum MSE values (training and validation) was used to determine the optimal model architecture. The average values of MSE in CV and training data for the established GRNN-QSTR model were 0.87 and 0.19, respectively. The Y-randomization results for the developed QSTR model derived using the 5-fold CV procedure yielded R2 values of