ADMET Evaluation in Drug Discovery. Part 17: Development of

Subscriber access provided by CORNELL UNIVERSITY LIBRARY

Article

ADMET Evaluation in Drug Discovery. 17. Development of Quantitative and Qualitative Prediction Models for Chemical-Induced Respiratory Toxicity Tailong Lei, Fu Chen, Hui Liu, Huiyong Sun, Yu Kang, Dan Li, Youyong Li, and Tingjun Hou Mol. Pharmaceutics, Just Accepted Manuscript • Publication Date (Web): 09 Jun 2017 Downloaded from http://pubs.acs.org on June 11, 2017

Just Accepted “Just Accepted” manuscripts have been peer-reviewed and accepted for publication. They are posted online prior to technical editing, formatting for publication and author proofing. The American Chemical Society provides “Just Accepted” as a free service to the research community to expedite the dissemination of scientific material as soon as possible after acceptance. “Just Accepted” manuscripts appear in full in PDF format accompanied by an HTML abstract. “Just Accepted” manuscripts have been fully peer reviewed, but should not be considered the official version of record. They are accessible to all readers and citable by the Digital Object Identifier (DOI®). “Just Accepted” is an optional service offered to authors. Therefore, the “Just Accepted” Web site may not include all articles that will be published in the journal. After a manuscript is technically edited and formatted, it will be removed from the “Just Accepted” Web site and published as an ASAP article. Note that technical editing may introduce minor changes to the manuscript text and/or graphics which could affect content, and all legal disclaimers and ethical guidelines that apply to the journal pertain. ACS cannot be held responsible for errors or consequences arising from the use of information contained in these “Just Accepted” manuscripts.

Molecular Pharmaceutics is published by the American Chemical Society. 1155 Sixteenth Street N.W., Washington, DC 20036 Published by American Chemical Society. Copyright © American Chemical Society. However, no copyright claim is made to original U.S. Government works, or works produced by employees of any Commonwealth realm Crown government in the course of their duties.

Page 1 of 43

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Molecular Pharmaceutics

ADMET Evaluation in Drug Discovery. 17. Development of Quantitative and Qualitative Prediction Models for Chemical-Induced Respiratory Toxicity Tailong Lei,a Fu Chen,a Hui Liu,a Huiyong Sun,a Yu Kang,a Dan Li,a Youyong Li,c Tingjun Houa,b,*

a

College of Pharmaceutical Sciences, Zhejiang University, Hangzhou, Zhejiang

310058, P. R. China; bState Key Lab of CAD&CG, Zhejiang University, Hangzhou, Zhejiang 310058, P. R. China; cInstitute of Functional Nano and Soft Materials (FUNSOM), Soochow University, Suzhou, Jiangsu 215123, P. R. China.

Corresponding author Tingjun Hou E-mail: [email protected] or

[email protected]

Phone: +86-571-88208412

1

ACS Paragon Plus Environment


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Table of Contents Graphic

2


Page 2 of 43

Page 3 of 43

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Abstract As a dangerous endpoint, respiratory toxicity can cause serious adverse health effects and even death. Meanwhile, it is a common and traditional issue in occupational and environmental protection. Pharmaceutical and chemical industries have a strong urge to develop precise and convenient computational tools to evaluate the respiratory toxicity of compounds as early as possible. Most of the reported theoretical models were developed based on the respiratory toxicity datasets with one single symptom, such as respiratory sensitization, and therefore these models may not afford reliable predictions for toxic compounds with other respiratory symptoms, such as pneumonia or rhinitis. Here, based on a diverse dataset of mouse intraperitoneal respiratory toxicity characterized by multiple symptoms, a number of quantitative and qualitative predictions models with high reliability were developed by machine learning approaches. First, a four-tier dimension reduction strategy was employed to find an optimal set of twenty molecular descriptors for model building. Then, six machine learning approaches were used to develop the prediction models, including relevance vector machine (RVM), support vector machine (SVM), regularized random forest (RRF), eXtreme gradient boosting (XGBoost), naïve Bayes (NB) and linear discriminant analysis (LDA). Among all of the models, the SVM regression model shows the most accurate quantitative predictions for the test set (q2ext =0.707), and the XGBoost classification model achieves the most accurate qualitative predictions for the test set (MCC of 0.644, AUC of 0.893 and global accuracy of 82.62%). The application domains were analyzed, and all of the tested compounds fall within the application domain coverage. We also examined the structural features of the compounds and important fragments with large prediction errors. In conclusion, the SVM regression model and the XGBoost classification model can be employed as accurate prediction tools for respiratory toxicity. Keywords: Quantitative Structure-Activity Relationship, Respiratory System Toxicity, Machine Learning, Dimension Reduction, eXtreme Gradient Boosting

3



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Introductions Safety and efficacy are the main factors that impede potential drug candidates into market.1 It has been estimated that about one third of the failed drug development programs could be attributed to preclinical toxicity and clinical adverse drug reaction.2 Among these safety issues, respiratory toxicity has become main causes of drug withdrawal.2-4 Drug-induced respiratory toxicity is usually underdiagnosed because it may not have distinct early signs or symptoms in common medications5-7 and can occur with significant morbidity and mortality.7-10 Therefore, careful surveillance and treatment of respiratory toxicity is of great importance. Among various types of drug-induced respiratory injuries that occur through direct cytotoxic oxidative stress or immune-mediated reactions, interstitial lung diseases and respiratory sensitization are the most common ones.6, 7, 11, 12 Cardiac medications, chemotherapeutic agents, anti-rheumatic agents, immunosuppressive agents, narcotics and antibiotics are the most common drugs of respiratory toxicity.12-15 For example, nitrofurantoin, an antibiotic, can cause dry cough, chest pain, dyspnea, hypoxemia, and pneumonia. Antineoplastics, such as bleomycin and nitrosoureas, can cause diffuse alveolar damage, bronchiolitis obliterans organizing pneumonia, and even pulmonary fibrosis. Amiodarone, a cardiac drugs, can cause diffuse alveolar damage and interstitial pneumonia. Nonsteroidal anti-inflammatory drugs may cause eosinophilic pneumonia. Anticoagulants, such as penicillamine and amphotericin B, may even cause pulmonary hemorrhage. All drugs implicated in pulmonary toxicity together with their adverse event frequencies can be found in the Pneumotox database.12 Moreover, in many industrialized countries, respiratory toxicity is a widespread occupational and environmental issue that causes serious medical and socioeconomic consequences.16-20 Workers from food, textile, dye and many other chemical industries may be threatened by chemicals that have respiratory toxicity.21-23 Among all occupational exposures, asthma is the most frequent outcome.24 Possible causative toxins of occupational respiratory diseases have been described by literatures.11, 25, 26 4


Page 4 of 43

Page 5 of 43

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


The most famous example is paraquat, a widely used herbicide that can cause acute pulmonary fibrosis and failure. Moreover, many chemical warfare agents, such as sulfur mustard, can cause very severe respiratory injuries.27, 28 Through a lot of efforts, respiratory toxic substances suspected in German occupational safety surveillance have been compiled into a public database.29 Respiratory sensitization and irritation caused by occupational respiratory toxicity issue may lead to severely physical disabilities. Unfortunately, few chemicals have adequate assessments of their respiratory tract risk before marketing. Identifying and characterizing respiratory sensitizers or allergens is a considerable challenge for toxicologists. There is no standard or guideline on systematic testing of respiratory toxicity30, and the only widely used guidelines are about respiratory sensitization for occupational protection and chemical labelling.31, 32 The traditional local lymph node assays and cytokine fingerprinting were used to recognize respiratory sensitizer, but they are expensive.30, 33, 34 Due to animal welfare and cost benefit, more and more alternative methods of toxicity assessment have been applied by the U.S. National Toxicology Program (NTP) and European Commission (EC) nowadays.35, 36 Some in vivo and in silico methods have been developed for the assessment of acute toxicity, eye irritation, endocrine disruption, and some other toxicity endpoints, but due to uncertain mechanism and various allergenic routes, no officially validated in vivo or in silico method has been reported for the assessment of respiratory toxicity.30, 33 Compared with experimental approaches, in silico methods offer a rapid and cheap screening platform for identifying the most likely candidate respiratory toxins from a list of chemicals. Till now, a number of quantitative structure-activity relationship (QSAR) models of respiratory sensitization have been developed (Table 1).37-48 It should be noted that the QSAR models for the complicated sensory irritation endpoints of volatile organic compounds, such as nasal pungency, upper respiratory tract irritation, inhalation anesthesia, etc.,49-51 are not listed in Table 1. Most of the reported in silico models rely upon chemical structural alerts or chemical group reactivity,52-54 and they are all empirical in nature and only consider organic agents with low molecular weight that 5



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

cause respiratory sensitization. Moreover, most of the reported QSAR models were built based on limited number of compounds (less than 1000) for the prediction of respiratory sensitization, and some substances that can cause other respiratory symptoms, such as pneumonia or rhinitis, were omitted. Thus these models may have limited practical application domain because respiratory toxicity is associated with many symptoms.16, 17, 20, 55 Therefore, in order to develop more reliable prediction models for respiratory toxicity, experimental data associated with more respiratory symptoms or endpoints, rather than respiratory sensitization, should be used. In this study, an extensive mouse intraperitoneal respiratory toxicity dataset with 1403 compounds was used to develop prediction models. After dimension reduction, six machine learning approaches, including relevance vector machine (RVM), support vector machine (SVM), regularized random forest (RRF), eXtreme gradient boosting (XGBoost), naïve Bayes (NB) and linear discriminant analysis (LDA), were utilized to establish the regression and classification models for the prediction of intraperitoneal respiratory toxicity in mouse. The reliability of the models was validated by internal and external datasets. Moreover, the structural features of the compounds and important fragments with large prediction errors were analyzed.

Methods and Materials 1. Preparation of Dataset In this study, a mouse intraperitoneal respiratory toxicity dataset of 2490 compounds was collected from the ChemIDplus public database56 of TOXNET databases where the Effect option was set to “LUNGS, THORAX or RESPIRATION”. The respiratory toxicity for these compounds was expressed as LD50 that is the dose required to kill half of total treated animals. The data quality was carefully verified. The inorganic compounds, metalorganic compounds, polymers and salts were eliminated. At last, 1403 organic compounds with experimental toxicity data were included in the dataset. The 3-D structures of the 1403 compounds in the dataset were generated from the SMILES representations and optimized in Discovery Studio 2.5 (DS2.5) molecular 6


Page 6 of 43

Page 7 of 43

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


simulation package.57 The whole dataset was randomly split into a training set with 935 compounds (66.6%) and an external test set with 468 compounds (33.4%). For building the regression models, the toxicity data were transformed into negative logarithm form as -log[mol/kg] (or pLD50). For building the classification models, according to the regulation of U.S. Environmental Protection Agency,58 the compounds

were

categorized

into

two

classes:

respiratory

toxicants

(LD50500mg/kg). The QSAR modeling pipeline of this study was shown in Figure 1.

2. Calculation of Molecular Descriptors A total of 334 molecular descriptors to characterize the physicochemical properties, structural representations, and drug-like properties of the studied compounds were generated by using the Molecular Operating Environment (MOE) molecular simulation package (version 2009).59 The descriptors that have all zero values or zero variance were removed. Then, the correlations across all pairs of descriptors were calculated, and the redundant descriptors with the correlation (r) higher than the predefined threshold (0.90) to any descriptor were also removed. Finally, 167 descriptors were chosen for further dimension reduction.

3. Dimension Reduction Dimension reduction is an essential step to select feature subset for model construction of high-dimensional data. The goal of dimension reduction is to remove redundant or irrelevant features without much loss of information. Filter approaches are the most common algorithms that use a specific statistics to rank features and sort through a threshold. Wrapper approaches are stochastic algorithms to select features by finding a global optimum of multiple models through adding or removing features. In this study, a four-tier strategy was utilized for dimension reduction. At first, the values of the descriptors were scaled and centralized, and then the modified chi-squared scoring was applied to filter descriptors. After that, the descriptors were pre-screened by using a simple univariate statistical method (rfSBF). To elaborate this 7



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

procedure, 10-fold cross-validated random forests were fitted based on the pre-filtered descriptor subset. For each iteration of resampling, a descriptor subset was pre-filtered by the generalized additive model between a single descriptor and the outcome using a smoothing spline basis function. The p-values provided by the models represent the significance scores of the descriptors, and the criterion to filter out the descriptors is less than or equals to 0.05. Subsequently, the recursive feature elimination (RFE) method incorporated with RF was used to select feature descriptors. This algorithm needs to evaluate and update the ranking criterion and eliminate the least important feature at each step. It was initially proposed by Guyon et al.60 by using SVM as its model function and then RF was introduced61, 62. The RFE-RF algorithm can be summarized as follows: 1. train a random forest model; 2. compute the permutated feature importance criterion; 3. wash out the least relevant variable; 4. repeat steps 1 to 3 until no further variables remain. After completing these procedures, the subset of descriptors that give the best prediction accuracy are identified. All the procedures of dimension reduction were implemented in R software (version 3.2.2 x64).

4. QSAR Modeling by Machine Learning Approaches. Machine learning approaches have been used to develop regression and classification models for the prediction of respiratory toxicity.63-66 Here, six machine learning methods, including RVM, SVM, RRF, XGBoost, NB and LDA, were employed for model building. The optimal parameters were determined by using the grid search method.67 A main package in R (version 3.2.2 x64), caret68, provides generic and object-oriented interfaces to the implementation of the following machine learning methods with good scalability. Relevance Vector Machine (RVM). RVM, inspired by Tipping, is a sparse Bayesian learning algorithm for regression and probabilistic classification derived from the standard SVM.69, 70 It shows improved generalization ability than SVM, and allows no free parameters. RVM is based on the principle of structural risk minimum, and conquers the Mercer condition of kernel functions. The first step of this algorithm 8


Page 8 of 43

Page 9 of 43

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


builds a hyper-parameter based zero-mean Gaussian prior distribution over the weights. And the second step reduces the less relevant vectors iteratively by integrating the weights to obtain the marginal likelihood of the posterior distribution. After that we can make predictions based on the sparse posterior distribution over the weights. In this study, the Gaussian radial basis function (RBF) kernel , = exp (−

) and the Laplacian kernel , = exp −

were used as

the kernel functions. The RMV based on the RBF or Laplacian kernel is referred to as rbfRVM or lpRVM. Support Vector Machine (SVM). SVM is a popular supervised learning algorithm within the Vapnik-Chervonenkis framework71 for QSAR modeling72-74 It was originally developed for classification, and can also be used for regression. The objective of SVM is to find an optimal separating hyperplane that maximizes the sum of the minimal distances from the data points to the hyperplane. SVM is designed to define the sum of kernels as the measure of the relative nearness of each test point to the data points of discriminated categories. In the case of regression, a loss function is introduced to the hyperplane for precision control. In this study, the RBF and Laplacian kernels were used as the kernel functions. The SVM based on the RBF or Laplacian kernel was referred to as rbfSVM or lpSVM. Regularized Random Forest (RRF). Random forest is an ensemble learning method constructed by multiple decision trees and outputs the consensus predictions from individual trees.75, 76 It projects the training data into a random subset to fit individual trees or nodes. After training, predictions for test samples can be made by voting the majority categories of individual decision trees or by averaging the predictions from all the individual trees. RRF introduces a tree regularization framework into random forest.77 It penalizes the selection of new features for splitting nodes when its information gain is similar to the features used in previous splits. And a smaller regularization value results a larger penalty. New features are sequentially added only if they gain improvement upon all the current features. Thus, every tree in the forest would be guaranteed to possess a set of informative, but non-redundant 9



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

features. eXtreme Gradient Boosting (XGBoost). Gradient boosting algorithm is a machine learning meta-algorithm to construct an ensemble strong learner from weak learners such as decision trees78,

79

, and XGBoost is an efficient and distributed

system to scale up gradient tree boosting algorithms80. Gradient boosting develops the model by approximately minimizing a cost function in a steepest descent fashion iteratively, and weak learners are added to update the model after cost function estimation in each step. XGBoost retrofits tree learning algorithm for handling sparse data, raises a weighted quantile sketch for approximate optimization calculation, and designs a column block structure for parallel learning.80 Naïve Bayes (NB). As a popular algorithm in QSAR studies, NB is a simple probabilistic classifier based on applying Bayes' theorem with strong independence assumptions between the features.81-87 Despite its much simplified assumptions, NB works quite well in reality. It requires only a few training data for parameter estimation, and somehow is insensitive to the missing data. NB learns fast, and it can effectively conquer the curse of dimensionality. Here, each compound in the dataset was categorized into the respiratory toxic and the respiratory nontoxic class, and a vector x = ⟨x1, x2,..., xn⟩ represents the n features (molecular descriptors). Then, we can get the conditional probability as p(Ck|x)= p(Ck)p(x|Ck)/p(x), where Ck refers to a compound’s class, p(Ck|x) refers the posterior probability of the compound class, p(Ck) refers the prior probability of the training set, p(x|Ck) refers the likelihood probability that a compound has certain features given its class, and p(x) refers the marginal probability that the given features will occur in the dataset. In addition, as a bad estimator, the predictive probability outputs are not to be taken too seriously. Linear Discriminant Analysis (LDA). LDA is a widely used algorithm raised by Ronald Fisher.88, 89 It defines the separation between the distributions of two classes of observations to be the ratio of the variance between the classes to the variance within the classes. LDA works when the variables for each observation are independent, and it assumes that the conditional probability density functions of two classes are both unimodally and normally distributed. When the means and covariances of the class 10


Page 10 of 43

Page 11 of 43

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


are not known, either the maximum likelihood estimate or the maximum a posteriori can be used to estimate them.

5. Evaluation and Validation of the QSAR Models The reliability and predictivity of the developed models were assessed by the internal and external validations. The internal validation was given by the leave-one-out (LOO) cross-validation, and the external validation was given by the predictions to the test set. The fit goodness of each regression model was assessed by adjusted R2 ( ) and

cross-validation R2 coeﬃcient (q2) as shown in Equations 1 and 2. !"

＝1 − (1 − ) × !#" ＝1 − [(%%&/(( − ))]/[(%%+/(( − 1)]

, =

--.﹣/01---.

(1)

(2)

where R2 is the square of the Pearson correlation coefficient, p is the number of the parameters in the regression equation, SSE is the sum of squares of errors, SST is the total sum of the squared deviations of the dependent variable values from their means, and PRESS is the predictive residual sum of squares. The conventional coefficient of determination R2 (,34 ) was used to evaluate the

predictive power of each model on the external test set. The acceptability thresholds of q2 for the training set and ,34 for the test set were both set to ≥ 0.5. A model is over-fitted when the difference between and ,34 is higher than 0.3.90, 91

Moreover, other two parameters, mean absolute error (MAE) and root mean square error (RMSE), were used to evaluate the quality of each model.92, 93 Each classifier was assessed by the following parameters based on true positives (TP), true negatives (TN), false positives (FP) and false negatives (FN): sensitivity (SE), specificity (SP), concordance or global accuracy (GA), balanced accuracy (BA), precision or positive predictive value (PPV), negative predictive value (NPV), false positive rate (FPR), false discovery rate (FDR), false negative rate (FNR), detection rate (DR), F-measure (F), G-means (G), Cohen's kappa coefficient (κ) and Matthews correlation coefficient (MCC).81, 82, 93-95 11



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 12 of 43

./

(3)

%8 = .756/

.7

(4)

9: = ./5675.756/

(5)

%& =

./567

./5.7

;: = (%& + %8)/2

(6)

88= =

./56/

./

(7)

>8= =

.7567

.7

(8)

?8 = .756/ = 1 − %8

6/

(9)

?@ = ./56/ = 1 − 88=

6/

(10)

?> =

(11)

67

./567

= 1 − %&

./

@ = ./5675.756/

(12)

? = "///A5"/-1

(13)

9 = √%& × %8

(14)

C=

(DEFHE)×(DEFHG)F(DGFHE)×(DGFHG) DEFDG DEFHGFDGFHE (DEFHGFDGFHE) (DEFHE)×(DEFHG)F(DGFHE)×(DGFHG) " (DEFHGFDGFHE)

IJJ =

./×.76/×67

K(./56/)×(./567)×(.756/)×(.7567)

(15) (16)

Cohen's kappa coefficient was used to measure inter-rater agreement for classification. Cohen's kappa coefficient and MCC range from -1 to 1, and a perfect classification gives a value of 1 while a random classification gives a value of 0. In addition, the classification capability was measured by the area under the receive operating characteristic (ROC) curve (AUC), which is a graphical plot to illustrate the classification performance by changing its discrimination threshold.

6. Analysis of Application Domain (AD) Because the training set for QSAR modelling may not cover the entire chemical space, the predictive and interpretable ability of any model to the query chemicals are limited. Therefore, the AD for any model should be defined.96 As a result, only a 12


Page 13 of 43

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


certain fraction of the query chemicals fall within the AD or coverage of the interpolation space, and only the predictions for such chemicals are reliable. In this study, the Standard Deviation Distance to Model (STD-DM) approach was used to estimate the AD of each model. The detailed description of the algorithm to define AD shown in Equation 17 has been described in previous literatures.97-100 %+@ − @I(L) = %+@&=M(L) = N

∑(P(Q)PR(Q)) 7"

(17)

where y(J) is a quantitative value of prediction for compound J, and N is the total number of the compounds in the test set. The margin range of AD is defined as three times of the STD-DM value.97 When a compound is outside the AD, the STD-DM value is high and accordingly the margin range is also high. Meanwhile, the Hotelling’s test and the associated leverage statistics were employed to determine the AD.101, 102 The leverage value hi measures the distance between the ith compound and the centroid of its training set. It is calculated from the descriptor matrix(X) as follows: ℎ = . (U . U)"

(18)

where xi is a row vector of descriptors for the ith compound. Compounds in the training set have leverage values between 0 and 1. A threshold value (h*) is generally fixed at 3(p+1)/n, where p is the number of descriptors, and n is the compound number of the training set. A leverage value higher than h* suggests there is a significant difference between a compound and the others in the dataset. The Williams plot, a plot of standardized residuals versus leverage values, was utilized to intuitively confirm a visual image of the outliers. Among the outliers, the response outliers (Y outliers) show standardized residuals higher than 3.0. The structurally influential outliers (X outliers) are compounds with the leverage values higher than the threshold value (hi > h*) and relatively low standard deviation. Cook’s distance is used to estimate the influence of a single observation to the model, and is defined as follows: 3

W

@ = #5" ∙ "W

(18)

where X is the standard residual of the ith compound, p is the number of descriptors, 13



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

and ℎ is the leverage value of the ith compound. The cutoff of the Cook’s distance is defined as 4/(n-p-1), and the compounds with Cook’s distance higher than the cutoff value are spotted as highly influential points of the model. The AD used in this study is based on the regression analysis, and therefore it is only suitable for the quantitative models. The AD analysis was also implemented in R software (version 3.2.2 x64).

7. Scaffold Analysis of Compounds with Large Prediction Errors The scaffolds for the compounds with large prediction errors (MAE > 1.0) and false classifications were examined systematically. The scaffolds for each compound were characterized by four representations, including Murcko frameworks, ring assemblies, bridge assemblies, and the side chains attached to Murcko frameworks. Murcko frameworks developed by Bemis103 were primarily used to characterize the cyclic substructures of compounds. The definitions of these four scaffold representations have been described in previous studies.104, 105 The scaffolds were generated by using the Generate Fragments component in Pipeline Pilot 7.5. The frequency of each scaffold architecture was counted, and the results were sorted by the scaffold frequencies. Finally, for each scaffold with frequency equal or larger than 2, its numbers present in the training and test sets were counted.

Results and Discussion 1. Property Distributions of Mouse Intraperitoneal Respiratory Toxicity Data In our study, 1403 organic compounds with their respiratory toxicity data were collected from the ChemIDplus database55 for model development and validation. The concrete toxicological endpoints include focal fibrosis (pneumoconiosis), acute pulmonary edema, bronchiolar constriction, bronchiolar dilation, changes in pulmonary vascular resistance, chronic pulmonary edema, cyanosis, dyspnea, pleural thickening, respiratory depression, respiratory obstruction, respiratory stimulation, structural or functional change in trachea or bronchi, and other changes. After data splitting, the training and test sets include 935 and 468 compounds, 14


Page 14 of 43

Page 15 of 43

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


respectively (Figure 2). In this study, 120 molecular descriptors with non-zero variances and relatively low pairwise correlations were used to execute dimension reduction. As shown in Figure 3, the RMSE of the training set decreases gradually and reaches stability at 20 descriptors. Therefore, the 20 representative molecular descriptors chosen by dimension reduction (Table S1) were selected for further modeling. As shown in Figure 4A, the chemical space of the test set is roughly within the extent of the training set, and therefore it was feasible to validate the prediction performance and generalization ability of the QSAR models. In addition, the normalized and original distributions of these descriptors for the training set are shown in Figure 4A and 4B, respectively. They describe the chemical space of the training set, and can be regarded as a representation of AD. The twenty representative molecular descriptors shown in Table S1 include PEOE_RPC-, BCUT_PEOE_3, GCUT_SMR_0, GCUT_SLOGP_3, Q_RPC-, vsurf_HL2, reactive, vsa_acc,

vsa_pol,

KierA3,

PEOE_VSA+0,

Q_VSA_FPNEG,

E_str,

PEOE_VSA_PNEG, ASA-, vsurf_HB2, SMR_VSA0, KierFlex, BCUT_SLOGP_3 and

vsa_other.

Among

them,

PEOE_RPC-,

BCUT_PEOE_3,

Q_RPC-,

PEOE_VSA+0, PEOE_VSA_PNEG, Q_VSA_FPNEG and ASA- are related to the atomic distribution of partial charges. GCUT_SMR_0 and SMR_VSA0 characterize molecular refractivity, which measures electronic polarizability. GCUT_SLOGP_3 and BCUT_SLOGP_3 provide atomic contributions to hydrophobicity. ASA-, vsurf_HL2,

vsurf_HB2,

vsa_acc,

vsa_pol,

PEOE_VSA+0,

Q_VSA_FPNEG,

PEOE_VSA_PNEG, SMR_VSA0 and vsa_other are van der Waals surface areas or solvent accessible surface areas for charge, hydrogen-bonding and other properties. Besides, bond stretch potential energy (E_str), Kier molecular flexibility index (KierFlex), third alpha modified shape index (KierA3) and numbers of reactive groups (reactive) are also relatively important. In a word, partial charges, atomic polarity and molecular flexibility contributed largely to respiratory toxicity, suggesting that respiratory toxicity may be related to the reactivity and membrane permeability of a compound. This assumption received the support of literatures about respiratory and skin sensitization.41, 52, 53, 106-110 Apparently, all the twenty descriptors 15



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

illustrate different contributions to respiratory toxicity, and their importance is shown in Figure 5.

2. Comparison of Various Regression Models for Mouse Respiratory Toxicity After the twenty representative molecular descriptors were identified, four machine learning approaches, including RVM, SVM, RRF and XGBoost, were employed to develop regression models for mouse respiratory toxicity. The statistical results for the training and test sets given by the optimal regression models based on the twenty descriptors are summarized in Table 2. According to the internal validations for the training set and the predictions for the test set, the performances of the four approaches are quite different. Obviously, among all the regression models, the =0.707). The predictive rbfSVM model gives the best prediction for the test set (,34

performances of the other models are lpSVM > lpRVM > XGBoost > RRF > rbfRVM. =0.699) of the lpSVM model is slightly worse than that The prediction capability (,34

of the rbfSVM model. And the other models are much worse than the two SVM models. It is interesting to observe that lpRVM is much better than rbfRVM. Therefore, kernel functions have great impact on the prediction capability of the developed models, and to a specific dataset, comparison of different kernel functions is quite necessary to determine which one is the best choice. The RMSE and MAE show similar trends to ,34 . In summary, considering the overall statistics and

prediction accuracy, the SVM approach based on the RBF kernel was recommended to develop the regression models for predicting mouse respiratory toxicity. The scatter plots of the experimental pLD50 values versus the predicted values given by the rbfSVM model for the training and test sets are shown in Figure 6. The AD coverages for the models defined by the STD-DM approach are summarized in Table 2. All the models show 100% AD coverage defined by the STD-DM approach for the test set. Therefore, the property distributions of the test set fall within the scope of the training set. In this study, the AD was also determined by the leverage approach, and the Williams plot and the Cook's distance plot are shown in Figure 7. Hence, 41 compounds may be considered as the response outliers because 16


Page 16 of 43

Page 17 of 43

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


their leverage values are higher than the warning limit (h*=0.06). At the same time, there are 23 structurally influential outliers outside the predicted residual threshold. Otherwise, due to the Cook's distances (cutoff=0.00388) of the compounds in the training set, there are 31 influential compounds that may greatly distort the regression modelling results. Previously, there were a few regression models about respiratory toxicity, and the only existed models were based on relatively small datasets containing complicated sensory irritations.49-51 Their toxicological targets include not only upper airways but also eyes, and this is because the models were developed for construction industries. Due to the same reason, the chemicals they used to develop models are all volatile compounds with non-lethal activities. Unlike these models, our quantitative models used a larger dataset of various respiratory toxicants with lethal activities. The prediction accuracies of our models are inferior to the models of sensory irritations, but they are still acceptable for pure theoretic models. In addition, this is the first time to build regression models for only respiratory toxicity.

3. Comparison of Various Classification Models for Mouse Respiratory Toxicity Besides the regression models, the classification models were developed by using five machine learning approaches, including NB, LDA, SVM, RRF and XGBoost. The prediction statistics for the training and test sets given by the optimal classification models using the twenty descriptors are summarized in Table 3. According to the statistical results, the XGBoost model shows the best predictive performance, and the NB model shows the worst predictive performance. For unbalanced datasets, MCC is an appropriate measure of the quality of binary classifications, and it is a correlation coefficient between the observations and predictions essentially. By taking account of the MCC values and other performance measures, the predictive performances of all the classification models were ranked from the best to the worst as XGBoost > rbfSVM > lpSVM > RRF > LDA > NB. All the models were LOO cross-validated. Apparently, among all the classification models, the XGBoost model gives the best prediction for the test set (MCC=0.644). 17



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

This model has the second highest sensitivity of 82.24%, the highest specificity of 83.21%, and the highest global accuracy of 82.62% for the test set. The F1 measure, G-means, and Cohen’s κ of this model also show encouraging values, implicating the robustness and predictive capability of the model. The rbfSVM model is slightly worse than the XGBoost model due to its lower specificity, but it shows even higher sensitivity than the XGBoost model. Of course, the XGBoost model bears lower false detection rate and false omission rate, and therefore it is the first choice. The XGBoost approach adds the second order Taylor’s expansion of the cost function to approximate the objective function. Meanwhile regularization items were adopted to the objective function in order to reduce the system complexity and improve the generalization ability of the model. Thus the approach shows robust for the outliers, and the feature sparsity is automatically utilized when trees split. This may be the reason why the XGBoost approach achieved the best performance. Another example elsewhere is given by three datasets of different inhibitors.111 Compared to the previous models of respiratory sensitization (Table 1), our classification models used a larger dataset containing various respiratory toxicity endpoints, and achieved reasonably good predictions. Our qualitative models have a larger chemical space than the previous models, and can be also used to predict the toxic chemicals causing pneumonia, respiratory irritation, etc. In this way, fewer respiratory toxic chemicals would be omitted when they were predicted using in silico models. In summary, XGBoost is recommended to develop the classification models for the prediction of mouse intraperitoneal respiratory toxicity. The ROC curves given by the XGBoost model for the training and test sets are shown in Figure 8. The plots show appreciably large areas under the curves and thus imply that the XGBoost model is an ideal classifier.

4. Analysis of Compounds with Large Prediction Errors As mentioned in the regression model, the rbfSVM model has good predictive capability for the test set, but some compounds in the test set still cannot be well 18


Page 18 of 43

Page 19 of 43

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


predicted. The Williams plot and the Cook's distance plot were used to detect the outliers and great influential points. Here, the MAE value was used to analyze the outlying degree of these outliers. When MAE > 1.0 was used as the criterion, the MAEs of the compounds with large prediction errors given by the rbfSVM model in Table 2 range from 1.035 to 1.950 for the test set and from 1.019 to 3.894 for the training set. In total, nine compounds in the test set (Table 5) and thirty compounds in the training set could not be well predicted by the rbfSVM model in Table 2. We observe that the compounds with large prediction errors usually have more discrete pLD50 values higher than -1.7 or lower than -3.3 in the training and test sets, suggesting that the compounds with LD50 values lower than 50 mg/kg or higher than 1995 mg/kg may not be reliably predicted by the rbfSVM model. Among the nine compounds in the test set with large prediction errors (Table 5), most of them are largely steric (such as No. 4 and No. 7 in Table 5) and flexible (such as No. 3 and No. 5 in Table 5), except a very small compound – cyanide. These large compounds (No. 4 and No. 7 in the Table 5) may be hard to reach the natural state in bodies and then have a drift toward immune reactions112, 113. These compounds with polar groups (No. 2, No. 3 and No. 7 in the Table 5) may induce electrophilic reactions easily52-54, 114, 115. Thus some descriptors related to polarity, hydrophobicity and charges of the partial surface of compounds may be inaccurate. The thirty compounds in the training set with large prediction errors have the same state. And they show diverse toxicity values and scaffolds (they have more diverse Murcko scaffolds according to the Pipeline Pilot calculation).

5. Analysis of Important Fragments For Misclassified Compounds by XGBoost We examined the molecular fragments for the compounds misclassified (false positives and false negatives) by the XGBoost classifier, and the important fragments are summarized in Table S2. Most of these important fragments contain heterocycles with nitrogen, oxygen or sulfur atoms. The reason is not well known. It is possible that these important fragments extracted from the misclassified compounds in the test set are not enriched in the compounds of the training set, such as No. 20, No. 25, No. 19



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

32, and No. 44 fragments in Table S2. Therefore, in order to enhance the prediction accuracy of the XGBoost classifier the addition of more compounds with these fragments shown in Table S2 to the training set is quite necessary. The misclassification of some compounds may be explained by the presence of a few activity cliffs (similar structures and dissimilar activity) or scaffold hops (dissimilar structures and similar activity) that distort the models. These phenomena have been observed in many previous studies.116-120 In this study, some compounds also exhibit these phenomena. As shown in Table 6, No. 3 and No.5 compounds are pairs of scaffold hops, while No. 1 and No. 2 compounds, No. 2 and No.3 compounds, No. 4 and No.5 compounds, No. 5 and No.6 compounds, No. 7 and No.8 compounds and No. 8 and No.9 compounds are all pairs of activity cliffs. Although the two compounds in each pair have similar structures, they belong to different class. No. 1, No. 2, No. 4, No. 5, No. 7 and No.8 compounds were classified correctly, but No. 3, No. 6 and No.9 compounds were misclassified. It is possible that some specific groups and their steric positions in the compounds may influence the toxicity of the compounds remarkably. To solve these problems, molecular structural fingerprints or more precise descriptors representing group connectivity and adjacency are suggested to be introduced for QSAR modeling. At last, the exact LD50 values of some compounds are not given, which may also cause incorrect predictions. For example, citronellal (CAS No.: 106-23-0), an edible essence with a LD50 value > 200 mg/kg, was misclassified (false negative). Thus more precise toxicity data should be used to provide more accurate toxicity information of compounds.

Conclusions In this study, based on a diverse dataset with a series of toxicological endpoints of mouse intraperitoneal respiratory toxicity, six regression models and six classification models were developed. A four-tier dimension reduction was utilized to determine an optimal subset of twenty molecular descriptors. Then, four machine learning approaches were used to build the regression models for respiratory toxicity. 20


Page 20 of 43

Page 21 of 43

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Considering the overall prediction accuracy for the test set, the rbfSVM model of 0.743 for the training set and ,34 of outperforms the others, and reaches

0.707 for the test set. In addition, five machine learning approaches were employed to develop the classification models for respiratory toxicity. The performance of the XGBoost approach is better than those of the other approaches. The optimal XGBoost model achieves MCC of 0.644, AUC of 0.893, sensitivity of 82.24%, specificity of 83.21%, and global accuracy of 82.62% for the test set. We also examined the structures of the compounds with large prediction errors in the regression analysis and the important fragments found in the misclassified compounds. According to the results of this study, we believe that the rbfSVM regression model and the XGBoost classification model can be employed for accurate quantitative and qualitative prediction of chemical-induced respiratory toxicity.

Acknowledgment This study was supported by the National Science Foundation of China (21575128; 81302679), the National Major Basic Research Program of China (2016YFB0201700 and 2016YFA0501701) and the Fundamental Research Funds for the Central Universities. We would like to thank the U.S. National Library of Medicine for valuable dataset of mouse intraperitoneal respiratory toxicity.

Abbreviations LD50, median lethal dose; QSAR, quantitative structure−activity relationship; OECD, the Organization for Economic Co-operation and Development; EPA, the U.S. Environmental Protection Agency; NTP, the U.S. National Toxicology Program; EC, European Commission; LOO-CV, leave-one-out cross-validated; AD, applicability domain; MOE, Molecular Operating Environment; RFE, recursive feature elimination; LDA, linear discriminant analysis; NB, naïve Bayes; RRF, regularized random forest; rbf, Gaussian radial basis kernel; lp, Laplacian kernel; SVM, support vector machine; RVM, relevance vector machine; XGBoost, eXtreme gradient boosting; R2, 21



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 22 of 43

coefficient of determination; MAE, mean absolute error; RMSE, root-mean-square error; TP, true positive; TN, true negative; FP, false positive; FN, false negative; SE, sensitivity; SP, specificity; GA, global accuracy or concordance; BA, balanced accuracy; PPV, positive predictive value or precision; NPV, negative predictive value; FPR, false positive rate; FDR, false discovery rate; FNR, false negative rate; DR, detection rate; F, F1-measure; G, G-means; κ, Cohen's kappa coefficient; MCC, Matthews correlation coefficient; ROC, the receive operating characteristic curve; AUC, the area under the ROC; STD-DM, standard deviation distance to model; ADME, absorption, distribution, metabolism, and excretion; PCA, principal component analysis.

Supporting Information Table S1. The description of the twenty representative descriptors chosen by dimension reduction; Table S2. The representative scaffolds found in the misclassified compounds (23 false positives and 38 false negatives).

References 1.

Harrison, R. K.

Phase II and phase III failures: 2013–2015. Nat. Rev. Drug Discov. 2016, 15,

(12), 817-818. 2.

Hornberg, J. J.; Laursen, M.; Brenden, N.; Persson, M.; Thougaard, A. V.; Toft, D. B.; Mow, T. Exploratory toxicology as an integrated part of drug discovery. Part I: Why and how. Drug Discov. Today 2014, 19, (8), 1131-1136.

3.

Siramshetty, V. B.; Nickel, J.; Omieczynski, C.; Gohlke, B.-O.; Drwal, M. N.; Preissner, R. WITHDRAWN--a resource for withdrawn and discontinued drugs. Nucleic Acids Res 2016, 44, (D1), D1080-6.

4.

Cai, M. C.; Xu, Q.; Pan, Y. J.; Pan, W.; Ji, N.; Li, Y. B.; Jin, H. J.; Liu, K.; Ji, Z. L.

ADReCS:

an ontology database for aiding standardization and hierarchical classification of adverse drug reaction terms. Nucleic Acids Res. 2015, 43, (D1), D907-D913. 5.

Schwaiblmair, M.

Drug Induced Interstitial Lung Disease. The Open Respiratory Medicine

Journal 2012, 6, (1), 63-74. 6.

Müller, N. L.; White, D. A.; Jiang, H.; Gemma, A.

Diagnosis and management of

7.

Kubo, K.; Azuma, A.; Kanazawa, M.; Kameda, H.; Kusumoto, M.; Genma, A.; Saijo, Y.; Sakai,

drug-associated interstitial lung disease. Br. J. Cancer 2004, 91, S24-S30. F.; Sugiyama, Y.; Tatsumi, K.; Dohi, M.; Tokuda, H.; Hashimoto, S.; Hattori, N.; Hanaoka, M.; Fukuda, Y.

Consensus statement for the diagnosis and treatment of drug-induced lung injuries. 22


Page 23 of 43

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Respiratory Investigation 2013, 51, (4), 260-277. 8.

Dhokarh, R.; Li, G.; Schmickl, C. N.; Kashyap, R.; Assudani, J.; Limper, A. H.; Gajic, O. Drug-Associated Acute Lung Injury. Chest 2012, 142, (4), 845-850.

9.

Amar, R. K.; Jick, S. S.; Rosenberg, D.; Maher, T. M.; Meier, C. R.

Drug-/radiation-induced

interstitial lung disease in the United Kingdom general population: Incidence, all-cause mortality and characteristics at diagnosis. Respirology 2012, 17, (5), 861-868. 10.

Demedts, M.; Wells, A. U.; Anto, J. M.; Costabel, U.; Hubbard, R.; Cullinan, P.; Slabbynck, H.; Rizzato, G.; Poletti, V.; Verbeken, E. K.; Thomeer, M. J.; Kokkarinen, J.; Dalphin, J. C.; Taylor, A. N.

Interstitial lung diseases: An epidemiological overview. Eur. Respir. J. 2001, 32, (18),

2s-16s. 11.

Beckett, W. S.

Occupational Respiratory Diseases. N. Engl. J. Med. 2000, 342, (6), 406-413.

12.

Camus,

Pneumotox

P.

online:

the

drug-induced

lung

diseases

website.

http://www.pneumotox.com/ (June 20, 2016), 13.

Camus, P.; Fanton, A.; Bonniaud, P.; Camus, C.; Foucher, P.

Interstitial Lung Disease Induced

by Drugs and Radiation. Respiration 2004, 71, (4), 301-326. 14.

Cooper, J. A., Jr.; White, D. A.; Matthay, R. A.

Drug-induced pulmonary disease. Part 2:

Noncytotoxic drugs. Am. Rev. Respir. Dis. 1986, 133, (3), 488-505. 15.

Cooper, J. A., Jr.; White, D. A.; Matthay, R. A.

Drug-induced pulmonary disease. Part 1:

Cytotoxic drugs. Am. Rev. Respir. Dis. 1986, 133, (2), 321-40. 16.

Tarlo, S. M.; Lemiere, C.

17.

Stevens, W. W.; Grammer, L. C., III.

Occupational Asthma. N. Engl. J. Med. 2014, 370, (7), 640-649. Occupational Rhinitis: an Update. Curr. Allergy. Asthm.

R. 2015, 15, (1), 487. 18.

Zhao, Y. A.; Shusterman, D.

Occupational Rhinitis and Other Work-Related Upper Respiratory

Tract Conditions. Clin. Chest Med. 2012, 33, (4), 637-647. 19.

Baur, X.; Sigsgaard, T.; Aasen, T. B.; Burge, P. S.; Heederik, D.; Henneberger, P.; Maestrelli, P.; Rooyackers, J.; Schlunssen, V.; Vandenplas, O.; Wilken, D.

Guidelines for the management of

work-related asthma. Eur. Respir. J. 2012, 39, (3), 529-545. 20.

Prezant, D. J.; Levin, S.; Kelly, K. J.; Aldrich, T. K.

Upper and lower respiratory diseases after

21.

Schyllert, C.; Rönmark, E.; Andersson, M.; Hedlund, U.; Lundbäck, B.; Hedman, L.; Lindberg,

occupational and environmental disasters. Mt. Sinai J. Med. 2008, 75, (2), 89-100. A.

Occupational exposure to chemicals drives the increased risk of asthma and rhinitis

observed for exposure to vapours, gas, dust and fumes: a cross-sectional population-based study. Occup. Environ. Med. 2016, 73, (10), 663-669. 22.

Doney, B.; Hnizdo, E.; Syamlal, G.; Kullman, G.; Burchfiel, C.; Martin, C. J.; Mujuru, P. Prevalence of Chronic Obstructive Pulmonary Disease Among US Working Adults Aged 40 to 70 Years. J. Occup. Environ. Med. 2014, 56, (10), 1088-1093.

23.

Hoppin, J. A.

Pesticides and respiratory health: where do we go from here? Occup. Environ.

Med. 2014, 71, (2), 80-80. 24.

Nemery, B., OCCUPATIONAL DISEASES | Overview A2 - Laurent, Geoffrey J. In Encyclopedia of Respiratory Medicine, Shapiro, S. D., Ed. Academic Press: Oxford, 2006; pp 186-191.

25.

McKay, C. A.

Toxin-induced Respiratory Distress. Emerg. Med. Clin. North Am. 2014, 32, (1),

127-147. 26.

Baur, X.

A compendium of causative agents of occupational asthma. J. Occup. Med. Toxicol. 23



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 24 of 43

2013, 8, (1), 15. 27.

Angelini, D. J.; Dorsey, R. M.; Willis, K. L.; Hong, C.; Moyer, R. A.; Oyler, J.; Jensen, N. S.; Salem, H.

Chemical warfare agent and biological toxin-induced pulmonary toxicity: could

stem cells provide potential therapies? Inhal. Toxicol. 2013, 25, (1), 37-62. 28.

Tang, F. R.; Loke, W. K.

Sulfur mustard and respiratory diseases. Crit. Rev. Toxicol. 2012, 42,

(8), 688-702. 29.

IFA.

GESTIS

database

on

hazardous

30.

Kimber, I.; Dearman, R. J.; Basketter, D. A.; Boverhof, D. R.

substances.

http://www.dguv.de/ifa/GESTIS/GESTIS-Stoffdatenbank/index-2.jsp (July 5, 2016), Chemical respiratory allergy:

Reverse engineering an adverse outcome pathway. Toxicology 2014, 318, 32-39. 31.

North, C. M.; Ezendam, J.; Hotchkiss, J. A.; Maier, C.; Aoyama, K.; Enoch, S.; Goetz, A.; Graham, C.; Kimber, I.; Karjalainen, A.; Pauluhn, J.; Roggen, E. L.; Selgrade, M.; Tarlo, S. M.; Chen, C. L.

Developing a framework for assessing chemical respiratory sensitization: A

workshop report. Regul. Toxicol. Pharmacol. 2016, 80, 295-309. 32.

Chapter 3.4 Respiratory or skin sensitization. Sixth revised ed.; United Nations Economic Commission for Europe: Geneva, 2015; p 149-157.

33.

Kimber, I.; Dearman, R. J.

What makes a chemical a respiratory sensitizer? Curr. Opin.

Allergy Clin. Immunol. 2005, 5, (2), 119-124. 34.

Karol, M.

Structure-activity relationships and computer-assisted analysis of respiratory

sensitization potential. Toxicol. Lett. 1996, 86, (2-3), 187-191. 35.

Casey, W.; Jacobs, A.; Maull, E.; Matheson, J.; Clarke, C.; Lowit, A.

A new path forward: The

Interagency Coordinating Committee on the Validation of Alternative Methods (ICCVAM) and National Toxicology Program's Interagency Center for the Evaluation of Alternative Toxicological Methods (NICEATM). J. Am. Assoc. Lab. Anim. 2015, 54, (2), 170-3. 36.

Rowlands, J. C.; Sander, M.; Bus, J. S.

FutureTox: Building the Road for 21st Century

37.

Dik, S.; Pennings, J. L. A.; van Loveren, H.; Ezendam, J.

Toxicology and Risk Assessment Practices. Toxicol. Sci. 2013, 137, (2), 269-277. Development of an in vitro test to

identify respiratory sensitizers in bronchial epithelial cells using gene expression profiling. Toxicol. In Vitro 2015, 30, (1), 274-280. 38.

Jarvis, J.; Seed, M. J.; Stocks, S. J.; Agius, R. M.

A refined QSAR model for prediction of

chemical asthma hazard. Occup. Med. 2015, 65, (8), 659-666. 39.

Dik, S.; Ezendam, J.; Cunningham, A. R.; Carrasquer, C. A.; van Loveren, H.; Rorije, E. Evaluation of in silico models for the identification of respiratory sensitizers. Toxicol. Sci. 2014, 142, (2), 385-394.

40.

Mekenyan, O.; Patlewicz, G.; Kuseva, C.; Popova, I.; Mehmed, A.; Kotov, S.; Zhechev, T.; Pavlov, T.; Temelkov, S.; Roberts, D. W.

A Mechanistic Approach to Modeling Respiratory

Sensitization. Chem. Res. Toxicol. 2014, 27, (2), 219-239. 41.

Enoch, S. J.; Seed, M. J.; Roberts, D. W.; Cronin, M. T. D.; Stocks, S. J.; Agius, R. M. Development of Mechanism-Based Structural Alerts for Respiratory Sensitization Hazard Identification. Chem. Res. Toxicol. 2012, 25, (11), 2490-2498.

42.

Seed, M.; Agius, R.

Further validation of computer-based prediction of chemical asthma

hazard. Occup. Med. 2010, 60, (2), 115-120. 43.

Sato, K.; Umemura, T.; Tamura, T.; Kusaka, Y.; Ido, T.; Aoyama, K.; Ueda, A.; Harada, K.; Minamoto, K.; Otsuki, T.

A respiratory sensitization study by a new quantitative 24


Page 25 of 43

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


structure-activity relationships (QSAR). Alternatives to Animal Testing and Experimentation 2010, 15, (3), 124-130. 44.

Jarvis, J.

Relationship between chemical structure and the occupational asthma hazard of low

molecular weight organic compounds. Occup. Environ. Med. 2005, 62, (4), 243-250. 45.

Cunningham, A. R.; Cunningham, S. L.; Consoer, D. M.; Moss, S. T.; Karol, M. H. Development of an information-intensive structure–activity relationship model and its application to human respiratory chemical sensitizers. SAR QSAR Environ. Res. 2005, 16, (3), 273-285.

46.

Karol, M. H.; Macina, O. T.; Cunningham, A.

Cell and molecular biology of chemical allergy.

Ann. Allergy, Asthma Immunol. 2001, 87, (6), 28-32. 47.

Graham, C.; Rosenkranz, H. S.; Karol, M. H.

Structure–Activity Model of Chemicals That

Cause Human Respiratory Sensitization. Regul. Toxicol. Pharmacol. 1997, 26, (3), 296-306. 48.

Seed, M. J.; Agius, R. M.

Progress with Structure–Activity Relationship modelling of

occupational chemical respiratory sensitizers. Curr. Opin. Allergy Clin. Immunol. 2017, 17, (2), 64-71. 49.

Abraham, M. H.; Sánchez-Moreno, R.; Gil-Lostes, J.; Acree, W. E.; Enrique Cometto-Muñiz, J.; Cain, W. S.

The biological and toxicological activity of gases and vapors. Toxicol. In Vitro

2010, 24, (2), 357-362. 50.

Nielsen, G. D.; Wolkoff, P.; Alarie, Y.

Sensory irritation: Risk assessment approaches. Regul.

51.

Doty, R. L.; Cometto-Muniz, J. E.; Jalowayski, A. A.; Dalton, P.; Kendal-Reed, M.; Hodgson, M.

Toxicol. Pharmacol. 2007, 48, (1), 6-18. Assessment of upper respiratory tract and ocular irritative effects of volatile chemicals in humans. Crit. Rev. Toxicol. 2004, 34, (2), 85-142. 52.

Ebbrell, D. J.; Madden, J. C.; Cronin, M. T. D.; Schultz, T. W.; Enoch, S. J.

Development of a

Fragment-Based in Silico Profiler for Michael Addition Thiol Reactivity. Chem. Res. Toxicol. 2016, 29, (6), 1073-1081. 53.

Dworak, J. J.; Roberts, D. W.; Calter, M. A.; Fields, C. A.; Borak, J.

Is Diacetyl a Respiratory

Sensitizer? A Reconsideration Using QSAR, QMM, and Competition Experiments. Chem. Res. Toxicol. 2013, 26, (5), 631-633. 54.

Enoch, S. J.; Roberts, D. W.; Cronin, M. T. D.

Mechanistic Category Formation for the

Prediction of Respiratory Sensitization. Chem. Res. Toxicol. 2010, 23, (10), 1547-1555. 55.

Rossi, S. E.; Erasmus, J. J.; McAdams, H. P.; Sporn, T. A.; Goodman, P. C.

Pulmonary Drug

Toxicity: Radiologic and Pathologic Manifestations. RadioGraphics 2000, 20, (5), 1245-1259. 56.

U.S. National Library of Medicine ChemIDplus database. https://chem.nlm.nih.gov/chemidplus/

57.

Discovery Studio 2.5 Guide, Accelrys Inc.: San Diego, CA, USA, 2009.

58.

U.S.

(November 15, 2015), EPA

40

CFR

156.64:

Toxicity

Category.

http://www.ecfr.gov/cgi-bin/text-idx?SID=01a7e4bc780b0368240af36919256d1c&mc=true&no de=se40.26.156_162&rgn=div84 (March 5, 2016), 59.

MOE molecular simulation package, Chemical Computing Group Inc.: Montreal, Candada, 2010.

60.

Guyon, I.; Weston, J.; Barnhill, S.; Vapnik, V.

Gene selection for cancer classification using

support vector machines. Mach. Learn. 2002, 46, (1/3), 389-422. 61.

Granitto, P. M.; Furlanello, C.; Biasioli, F.; Gasperi, F.

Recursive feature elimination with

25



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 26 of 43

random forest for PTR-MS analysis of agroindustrial products. Chemometrics Intellig. Lab. Syst. 2006, 83, (2), 83-90. 62.

Svetnik, V.; Liaw, A.; Tong, C.; Wang, T., Application of Breiman’s Random Forest to Modeling Structure-Activity Relationships of Pharmaceutical Molecules. In MCS 2004: Multiple Classifier Systems, Roli, F.; Kittler, J.; Windeatt, T., Eds. Springer Berlin Heidelberg: 2004; pp 334-343.

63.

Eklund, M.; Norinder, U.; Boyer, S.; Carlsson, L.

Choosing Feature Selection and Learning

Algorithms in QSAR. J. Chem. Inf. Model. 2014, 54, (3), 837-843. 64.

Tian, S.; Wang, J.; Li, Y.; Xu, X.; Hou, T.

Drug-likeness Analysis of Traditional Chinese

Medicines: Prediction of Drug-likeness Using Machine Learning Approaches. Mol. Pharm. 2012, 9, (10), 2875-2886. 65.

Chen, L.; Li, Y.; Yu, H.; Zhang, L.; Hou, T.

Computational models for predicting substrates or

inhibitors of P-glycoprotein. Drug Discov. Today 2012, 17, (7-8), 343-351. 66.

Hou, T.; Wang, J.

Structure – ADME relationship: still a long way to go? Expert Opin. Drug

Metab. Toxicol. 2008, 4, (6), 759-770. 67.

Cortez, P., Modern Optimization with R. In Use R!, Springer: New York, 2014.

68.

Kuhn, M.

Building Predictive Models in R Using the caret Package. J. Stat. Softw. 2008, 28,

(5), 1-26. 69.

Burden, F. R.; Winkler, D. A.

Relevance Vector Machines: Sparse Classification Methods for

QSAR. J. Chem. Inf. Model. 2015, 55, (8), 1529-1534. 70.

Tipping, M. E.

Sparse Bayesian learning and the relevance vector machine. J. Mach. Learn.

Res. 2001, 1, (3), 211-244. 71.

Cortes, C.; Vapnik, V.

72.

Zhou, S.; Li, G.-B.; Huang, L.-Y.; Xie, H.-Z.; Zhao, Y.-L.; Chen, Y.-Z.; Li, L.-L.; Yang, S.-Y.

Support-vector networks. Mach. Learn. 1995, 20, (3), 273-297.

A prediction model of drug-induced ototoxicity developed by an optimal support vector machine (SVM) method. Comput. Biol. Med. 2014, 51, 122-127. 73.

Hou, T.; Wang, J.; Li, Y.

ADME Evaluation in Drug Discovery. 8. The Prediction of Human

Intestinal Absorption by a Support Vector Machine. J. Chem. Inf. Model. 2007, 47, (6), 2408-2415. 74.

Hou, T.; Li, Y.; Zhang, W.; Wang, J.

Recent Developments of In Silico Predictions of Intestinal

Absorption and Oral Bioavailability. Combinatorial Chemistry & High Throughput Screening 2009, 12, (5), 497-506. 75.

Sheridan, R. P.

Using Random Forest To Model the Domain Applicability of Another Random

Forest Model. J. Chem. Inf. Model. 2013, 53, (11), 2837-2850. 76.

Svetnik, V.; Liaw, A.; Tong, C.; Culberson, J. C.; Sheridan, R. P.; Feuston, B. P.

Random

Forest: A Classification and Regression Tool for Compound Classification and QSAR Modeling. J. Chem. Inf. Comput. Sci. 2003, 43, (6), 1947-1958. 77.

Deng, H.; Runger, G.

Gene selection with guided regularized random forest. Pattern Recogn

2013, 46, (12), 3483-3489. 78.

Singh, K. P.; Gupta, S.

In silico prediction of toxicity of non-congeneric industrial chemicals

using ensemble learning based modeling approaches. Toxicol. Appl. Pharmacol. 2014, 275, (3), 198-212. 79.

Friedman, J. H.

Greedy function approximation: A gradient boosting machine. Ann Stat 2001,

29, (5), 1189-1232. 26


Page 27 of 43

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


80.

Chen, T.; Guestrin, C., XGBoost: A Scalable Tree Boosting System. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD'16, Association for Computing Machinery (ACM): New York, NY, USA, 2016; pp 785-794.

81.

Shi, H.; Tian, S.; Li, Y.; Li, D.; Yu, H.; Zhen, X.; Hou, T.

Absorption, Distribution,

Metabolism, Excretion, and Toxicity Evaluation in Drug Discovery. 14. Prediction of Human Pregnane X Receptor Activators by Using Naive Bayesian Classification Technique. Chem. Res. Toxicol. 2015, 28, (1), 116-125. 82.

Li, D.; Chen, L.; Li, Y.; Tian, S.; Sun, H.; Hou, T.

ADMET Evaluation in Drug Discovery. 13.

Development ofin SilicoPrediction Models for P-Glycoprotein Substrates. Mol. Pharm. 2014, 11, (3), 716-726. 83.

Wang, S.; Li, Y.; Wang, J.; Chen, L.; Zhang, L.; Yu, H.; Hou, T.

ADMET Evaluation in Drug

Discovery. 12. Development of Binary Classification Models for Prediction of hERG Potassium Channel Blockage. Mol. Pharm. 2012, 9, (4), 996-1010. 84.

Hand, D. J.; Yu, K.

85.

Tian, S.; Sun, H.; Li, Y.; Pan, P.; Li, D.; Hou, T.

Idiot's Bayes - Not so stupid after all? Int. Stat. Rev. 2001, 69, (3), 385. Development and Evaluation of an Integrated

Virtual Screening Strategy by Combining Molecular Docking and Pharmacophore Searching Based on Multiple Protein Structures. Journal of Chemical Information and Modeling 2013, 53, (10), 2743-2756. 86.

Tian, S.; Sun, H.; Pan, P.; Li, D.; Zhen, X.; Li, Y.; Hou, T.

Assessing an Ensemble

Docking-Based Virtual Screening Strategy for Kinase Targets by Considering Protein Flexibility. Journal of Chemical Information and Modeling 2014, 54, (10), 2664-2679. 87.

Wang, S.; Sun, H.; Liu, H.; Li, D.; Li, Y.; Hou, T.

ADMET Evaluation in Drug Discovery. 16.

Predicting hERG Blockers by Combining Multiple Pharmacophores and Machine Learning Approaches. Molecular Pharmaceutics 2016, 13, (8), 2855-2866. 88.

Luan, F.; Ma, W.; Zhang, X.; Zhang, H.; Liu, M.; Hu, Z.; Fan, B. T.

Quantitative

structure-activity relationship models for prediction of sensory irritants (logRD50) of volatile organic chemicals. Chemosphere 2006, 63, (7), 1142-1153. 89.

Fisher, R. A.

The use of multiple measurements in taxonomic problems. Annals of Eugenics

1936, 7, (2), 179-188. 90.

Tropsha, A.

Best Practices for QSAR Model Development, Validation, and Exploitation. Mol.

Inform. 2010, 29, (6-7), 476-488. 91.

Eriksson, L.; Jaworska, J.; Worth, A. P.; Cronin, M. T. D.; McDowell, R. M.; Gramatica, P. Methods for Reliability and Uncertainty Assessment and for Applicability Evaluations of Classification- and Regression-Based QSARs. Environ. Health Perspect. 2003, 111, (10), 1361-1375.

92.

Lei, T.; Li, Y.; Song, Y.; Li, D.; Sun, H.; Hou, T.

ADMET evaluation in drug discovery: 15.

Accurate prediction of rat oral acute toxicity using relevance vector machine and consensus modeling. J Cheminformatics 2016, 8, (1), 6. 93.

Roy, K.; Mitra, I.

On Various Metrics Used for Validation of Predictive QSAR Models with

Applications in Virtual Screening and Focused Library Design. Combinatorial Chem. High Throughput Screening 2011, 14, (6), 450-474. 94.

Das, R. N.; Roy, K.

Predictive modeling studies for the ecotoxicity of ionic liquids towards the

green algae Scenedesmus vacuolatus. Chemosphere 2014, 104, 170-176. 95.

Chang, C.-Y.; Hsu, M.-T.; Esposito, E. X.; Tseng, Y. J.

Oversampling to Overcome Overfitting:

27



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 28 of 43

Exploring the Relationship between Data Set Composition, Molecular Descriptors, and Predictive Modeling Methods. J. Chem. Inf. Model. 2013, 53, (4), 958-971. 96.

Weaver, S.; Gleeson, M. P.

The importance of the domain of applicability in QSAR modeling.

J. Mol. Graph. Model. 2008, 26, (8), 1315-1326. 97.

Kaneko, H.; Funatsu, K.

Applicability Domain Based on Ensemble Learning in Classification

and Regression Analyses. J. Chem. Inf. Model. 2014, 54, (9), 2469-2482. 98.

Sushko, I.; Novotarskyi, S.; Körner, R.; Pandey, A. K.; Kovalishyn, V. V.; Prokopenko, V. V.; Tetko, I. V.

Applicability domain for in silico models to achieve accuracy of experimental

measurements. J. Chemometr. 2010, 24, (3-4), 202-208. 99.

Sushko, I.; Novotarskyi, S.; Körner, R.; Pandey, A. K.; Cherkasov, A.; Li, J.; Gramatica, P.; Hansen, K.; Schroeter, T.; Müller, K.-R.; Xi, L.; Liu, H.; Yao, X.; Öberg, T.; Hormozdiari, F.; Dao, P.; Sahinalp, C.; Todeschini, R.; Polishchuk, P.; Artemenko, A.; Kuz’min, V.; Martin, T. M.; Young, D. M.; Fourches, D.; Muratov, E.; Tropsha, A.; Baskin, I.; Horvath, D.; Marcou, G.; Muller, C.; Varnek, A.; Prokopenko, V. V.; Tetko, I. V.

Applicability Domains for

Classification Problems: Benchmarking of Distance to Models for Ames Mutagenicity Set. J. Chem. Inf. Model. 2010, 50, (12), 2094-2111. 100.

Tetko, I. V.; Sushko, I.; Pandey, A. K.; Zhu, H.; Tropsha, A.; Papa, E.; Öberg, T.; Todeschini, R.; Fourches, D.; Varnek, A.

Critical Assessment of QSAR Models of Environmental Toxicity

againstTetrahymena pyriformis:Focusing on Applicability Domain and Overfitting by Variable Selection. J. Chem. Inf. Model. 2008, 48, (9), 1733-1746. 101.

Gramatica, P.

Principles of QSAR models validation: internal and external. QSAR Comb. Sci.

2007, 26, (5), 694-701. 102.

Jaworska, J.; Nikolova-Jeliazkova, N.; Aldenberg, T.

QSAR applicabilty domain estimation by

projection of the training set descriptor space: a review. ATLA-Altern. Lab. Anim. 2005, 33, (5), 445-59. 103.

Bemis, G. W.; Murcko, M. A.

The Properties of Known Drugs. 1. Molecular Frameworks. J.

Med. Chem. 1996, 39, (15), 2887-2893. 104.

Tian, S.; Wang, J.; Li, Y.; Li, D.; Xu, L.; Hou, T.

The application of in silico drug-likeness

predictions in pharmaceutical research. Adv Drug Deliver Rev 2015, 86, 2-10. 105.

Tian, S.; Li, Y.; Wang, J.; Xu, X.; Xu, L.; Wang, X.; Chen, L.; Hou, T.

Drug-likeness analysis

of traditional Chinese medicines: 2. Characterization of scaffold architectures for drug-like compounds, non-drug-like compounds, and natural compounds from traditional Chinese medicines. J Cheminformatics 2013, 5, (1), 5. 106.

Natsch, A.; Emter, R.

Reaction Chemistry to Characterize the Molecular Initiating Event in

107.

Enoch, S. J.; Roberts, D. W.; Madden, J. C.; Cronin, M. T.

Skin Sensitization: A Journey to Be Continued. Chem. Res. Toxicol. 2017, 30, (1), 315-331. Development of an in silico profiler

for respiratory sensitisation. ATLA-Altern. Lab. Anim. 2014, 42, (6), 367-75. 108.

Lalko, J. F.; Kimber, I.; Dearman, R. J.; Gerberick, G. F.; Sarlo, K.; Api, A. M.

Chemical

reactivity measurements: Potential for characterization of respiratory chemical allergens. Toxicol. In Vitro 2011, 25, (2), 433-445. 109.

Anderson, S. E.; Wells, J.; Fedorowicz, A.; Butterworth, L. F.; Meade, B.; Munson, A. E. Evaluation of the Contact and Respiratory Sensitization Potential of Volatile Organic Compounds Generated by Simulated Indoor Air Chemistry. Toxicol. Sci. 2007, 97, (2), 355-363.

110.

Rodford, R.; Patlewicz, G.; Walker, J. D.; Payne, M. P. 28


Quantitative structure-activity

Page 29 of 43

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


relationships for predicting skin and respiratory sensitization. Environ. Toxicol. Chem. 2003, 22, (8), 1855. 111.

Mustapha, I. B.; Saeed, F.

Bioactive Molecule Prediction Using Extreme Gradient Boosting.

Molecules 2016, 21, (8), 983. 112.

Meno, K. H.

113.

Malo, J.-L.; Chan-Yeung, M.

Allergen structures and epitopes. Allergy 2011, 66, 19-21. Agents causing occupational asthma. J. Allergy Clin. Immunol.

2009, 123, (3), 545-550. 114.

Wondrousch, D.; Böhme, A.; Thaens, D.; Ost, N.; Schüürmann, G. Local Electrophilicity Predicts the Toxicity-Relevant Reactivity of Michael Acceptors. J Phys Chem Lett 2010, 1, (10), 1605-1610.

115.

Enoch, S. J.; Roberts, D. W.; Cronin, M. T. D.

Electrophilic Reaction Chemistry of Low

Molecular Weight Respiratory Sensitizers. Chem. Res. Toxicol. 2009, 22, (8), 1447-1453. 116.

Dimova, D.; Bajorath, J.

Advances in Activity Cliff Research. Mol. Inform. 2016, 35, (5),

181-191. 117.

Stumpfe, D.; Dimova, D.; Bajorath, J.

Systematic assessment of scaffold hopping versus

activity cliff formation across bioactive compound classes following a molecular hierarchy. Biorg. Med. Chem. 2015, 23, (13), 3183-3191. 118. Greene, N.; Pennie, W.

Computational toxicology, friend or foe? Toxicol. Res.-UK 2015, 4, (5),

1159-1172. 119.

Bajorath, J.

Exploring Activity Cliffs from a Chemoinformatics Perspective. Mol. Inform.

2014, 33, (6-7), 438-442. 120.

Hu, X.; Hu, Y.; Vogt, M.; Stumpfe, D.; Bajorath, J.

MMP-Cliffs: Systematic Identification of

Activity Cliffs on the Basis of Matched Molecular Pairs. J. Chem. Inf. Model. 2012, 52, (5), 1138-1145.

29



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Legend of Figures Figure 1. Pipeline of QSAR modeling for the prediction of chemical-induced respiratory toxicity. Figure 2. Distributions of the experimental PLD50 values for the whole dataset (n = 1403; white bars), training set (n = 935; light gray bars) and test set (n = 468; black bars). Figure 3. The last dimension reduction procedure of molecular descriptors based on the recursive feature elimination incorporated with random forest (RFE-RF) method (10-fold cross validation). Figure 4. Comparison of the original (A) and normalized (B) distributions of the final selected descriptors in different datasets in this study. White boxplot stands for the training set and orange boxplot stands for the test set. Figure 5. The importance of molecular descriptors selected by RFE-RF based the decrease of node impurity (mean squared error). Figure6. Predicted results of the best regression model (rbfSVM). Scatter plots of the experimental pLD50 values versus the predicted values for the compounds in the (A) training and (B) test sets were given by the rbfSVM model in Table 2. Figure 7. Application domain defined in this study. Williams plot (A) and Cook's distance plot (B) were given using the leverage approach. Figure 8. Predicted results of the best classification model (lpSVM). The ROC curves for the (C) training and (D) test set were given by the lpSVM model in Table 3.

30


Page 30 of 43

Page 31 of 43

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Table 1. Reported Classification Models of Respiratory Sensitization No.

Method

Dataset

Source

Descriptors

Validation

Sensitivity

Specificity

Concordance

Ref

22 chemicals

Gene expression test

Gene signatures

LOO

SVM: 100%, RF: 92%, PAM: 83%

SVM: 90%, RF: 90%, PAM: 90%

SVM: 95%, RF: 91%, PAM: 86%

37

1

SVM, RF, Prediction Analysis for Microarrays in R (PAM) logistic regression

337 chemicals (303/34) 659 chemicals

Published literatures

Substructures

No data

90%

96%

38


No data

No data

33-100%; tier approach: 92%

32-100%; tier approach: 98%

The area under the ROC curve: 0.95 42-100%, best: Derek Nexus; tier approach: 96%

202 chemicals


Descriptors by MOPAC

No data

Pipeline with DPRA: 89%, pipeline without DPRA: 84%



40

186 chemicals 157 chemicals

Published literatures Australian Hazardous Substances Information System Published literature

Structural alerts

No data

91%

91%

No data

41

No data

79%

93%

The area under the ROC curve: 0.87

42

Descriptors by ADMEWorks

LOO

No data

No data

Linear learning machine: 95.75%, neural network: 95.33%, ILS-LDA: 95.33%, SVM: 97.2%, Adaboost: 96.73% pre-1995: 90%

43

2

39

5

MultiCASE, cat-SAR, Javis QSAR model, Enoch alerts, Derek Nexus, tier approach (DerekMultiCASE) OASIS pipeline with/without DPRA-lysine reactivity component Toxtree

6

logistic regression

7

Linear learning machine, neural network, ILS-LDA, SVM, Adaboost

214 chemicals

8

logistic regression


Substructure fragments

No data

pre-1995: 69%, post-1995: 86%

pre-1995: 93%, post-1995: 99%

cat-SAR


LOO

10

LDA

78 chemicals


ABC model: 94%, ABCH model: 89% 85.0%

ABC model: 87%, ABCH model: 95% 74.4%

ABC model: 91%, ABCH model: 92% No data

Bayesian combination of CASE/ MultiCASE

160 chemicals


CASE: 72-75%, MultiCASE: 80%, Bayesian combination: 95%

CASE: 95-98%, MultiCASE: 98%, Bayesian combination: 95%

CASE: 85%, MultiCASE: 89%, Bayesian combination: 95%

47

11

Fragments by Sybyl HQSAR Descriptors by Molecular Modeling Pro CASE/ MultiCASE fragments

45

9

477 chemicals (379/98) 80 chemicals

3

4

No data

No data

Abbreviations: RF, random forest; SVM, support vector machine; cat-SAR, categorical structure-activity relationship; ILS, iterative least square; LDA: linear discriminant analysis; DPRA, the direct peptide reactivity assay; ROC curve, receiver operating characteristic curve; LOO, leave-one-out cross validation.

31


44

46


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 32 of 43

Table 2. Statistical results for the regression models based on 20 descriptors for the training and test sets Models

R2adj

,YZZ

,34

RMSEtrain

RMSEtest

MAEtrain

MAEtest

AD Coverage

rbfRVM

0.626 0.971 0.743 0.899 0.929 0.967

0.622 0.970 0.743 0.899 0.926 0.963

0.593 0.676 0.707 0.699 0.624 0.664

0.480 0.135 0.400 0.266 0.270 0.114

0.477 0.418 0.398 0.406 0.454 0.427

0.338 0.100 0.199 0.111 0.159 0.088

0.350 0.305 0.287 0.290 0.230 0.303

100.0% 100.0% 100.0% 100.0% 100.0% 100.0%

lpRVM rbfSVM lpSVM RRF XGBoost

Table 3. Statistical results for the classification models based on 20 descriptors for the training and test sets Models

SE

SP

GA

BA

PPV

NPV

FPR

FDR

FNR

DR

F1

G

κ

MCC

AUC

a

65.98%

73.87%

69.14%

69.93%

79.13%

59.13%

26.13%

20.87%

34.02%

39.60%

0.720

0.723

0.383

0.390

0.786

b

TE

56.07%

70.80%

61.82%

63.44%

75.00%

50.79%

29.20%

25.00%

43.93%

34.19%

0.642

0.649

0.251

0.263

0.717

TR

82.91%

53.92%

71.32%

68.42%

72.98%

67.76%

46.08%

27.02%

17.09%

49.76%

0.776

0.778

0.381

0.387

0.759

TR NB

LDA TE

80.37%

56.93%

71.23%

68.65%

74.46%

65.00%

43.07%

25.54%

19.63%

49.00%

0.773

0.774

0.382

0.384

0.773

rbfSV

TR

99.53%

99.05%

99.34%

99.29%

99.37%

99.29%

0.95%

0.63%

0.47%

59.73%

0.994

0.994

0.986

0.986

1.000

M

TE

83.18%

79.56%

81.77%

81.37%

86.41%

75.17%

20.44%

13.59%

16.82%

50.71%

0.848

0.848

0.621

0.622

0.866

TR

99.84%

99.76%

99.81%

99.80%

99.84%

99.76%

0.24%

0.16%

0.16%

59.92%

0.998

0.998

0.996

0.996

1.000

lpSVM TE

81.78%

79.56%

80.91%

80.67%

86.21%

73.65%

20.44%

13.79%

18.22%

49.86%

0.839

0.840

0.605

0.606

0.884

TR

100.0%

100.0%

100.0%

100.0%

100.0%

100.0%

0.00%

0.00%

0.00%

60.02%

1.000

1.000

1.000

1.000

1.000

TE

79.91%

81.75%

80.63%

80.83%

87.24%

72.26%

18.25%

12.76%

20.09%

48.72%

0.834

0.835

0.602

0.606

0.869

XGBo

TR

100.0%

100.0%

100.0%

100.0%

100.0%

100.0%

0.00%

0.00%

0.00%

60.02%

1.000

1.000

1.000

1.000

1.000

ost

TE

82.24%

83.21%

82.62%

82.73%

88.44%

75.00%

16.79%

11.56%

17.76%

50.14%

0.852

0.853

0.642

0.644

0.893

RRF

a

b

TR represents the training set; TE represents the test set.

32


Page 33 of 43

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Table 4. The optimal parameters identified by the grid search for QSAR modeling Models

Optimal parameters rbfRVM

The kernel width σ = 0.0518014.

lpRVM

The kernel width σ = 0.1616710.

rbfSVM

Regression

lpSVM

RRF

The kernel width σ = 0.04102, the penalty parameter C = 16.00539, and ε in the loss function = 0.1. The kernel width σ = 0.14125, the penalty parameter C = 5.551986, and ε in the loss function = 0.1. The number of predictors at each split = 16, the number of trees = 500, regularization value = 0.174, importance coefficient = 0.7. The max number of boosting iterations = 95, maximum tree depth = 10, step

XGBoost

shrinkage = 0.21, minimum loss reduction = 0.14, subsample ratio of columns = 0.59, minimum sum of instance weight = 1

NB

No hyperparameters

LDA

No hyperparameters

rbfSVM

Classification

lpSVM

RRF

The kernel width σ = 0.294, the penalty parameter C = 20.326, and ε in the loss function = 0.1. The kernel width σ = 0.8996, the penalty parameter C = 2.2868, and ε in the loss function = 0.1. The number of predictors at each split = 16, the number of trees = 507, regularization value = 0.668050052, importance coefficient = 0.727091545. The max number of boosting iterations = 314, maximum tree depth = 13, step

XGBoost

shrinkage = 0.2664, minimum loss reduction = 0.1999, subsample ratio of columns = 0.365, minimum sum of instance weight = 1

33



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 34 of 43

Table 5. Experimental and predicted LD50 values for the nine tested compounds with large prediction errors (MAE > 1.0) No.

Structure

Experimental pLD50

SVM Predicted pLD50

MAE

1

-1.398

-3.348

1.950

2

-0.477

-1.992

1.515

3

-1.176

-2.663

1.487

4

-3.301

-2.072

1.229

5

-0.698

-1.924

1.225

6

-3.301

-2.107

1.194

7

-0.845

-2.008

1.163

8

-3.681

-2.528

1.153

9

-1.699

-2.734

1.035

34


Page 35 of 43

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Table 6. The examples of activity cliffs and scaffold hops in the test set Fragments

No.

Molecules

Name 4,5,6,7-tetrahydro-2-(2-

1

pyridinyl)-1H-isoindole1,3(2H)-dione 2-(2,5-dioxo-1-phenyl-3

2

-pyrrolidinyl)-1H-isoind ole-1,3(2H)-dione

3

CAS

LD50

No.

(mg/kg)

6135607-8

5260493-0

1'-(m-tolyl)-[1,3'-bipyrro

69557-

lidine]-2,2',5,5'-tetraone

05-7

1100

61.5

600

Effects

Respiratory stimulation

Respiratory depression


Classification

TN

FN

FP

3-(5-((dimethylamino)m 4

ethyl)-1,3,4-thiadiazol-2

74796-

-yl)-2-phenyl-4(3H)-qui

87-5

316


TP

nazolinone 2-(((2-methylphenyl)am ino)methyl)-3-(5-(((2-m 5

ethylphenyl)amino)meth yl)-1,3,4-thiadiazol-2-yl

7479695-5

681


TN

)-4(3H)-quinazolinone 3-(5-((ethylazo)methyl)6

1,3,4-thiadiazol-2-yl)-2-

10351

(2-phenylethenyl)-4(3H)

5-11-3

275


FN

-quinazolinone 1-(((5-nitro-2-furoyl)me 7

thylene)amino)-hydantoi n Tetrahydro-1-(((5-nitro-

8

2-furoyl)methylene)ami no)-2(1H)-pyrimidinone

9055677-7

9181891-6

110

1100



TP

TN

5-(morpholinomethyl)-3 9

-(((5-nitro-2-furoyl)met

92297-

hylene)amino)-2-oxazoli

09-1

dinone

35


64


FN


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Figure 1

36


Page 36 of 43

Page 37 of 43

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Figure 2

37



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Figure 3

38


Page 38 of 43

Page 39 of 43

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


(A)

(B)

Figure 4

39



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Figure 5

40


Page 40 of 43

Page 41 of 43

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Figure 6

(B)

(A)

41



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Figure 7

42


Page 42 of 43

Page 43 of 43

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Figure 9

43


ADMET Evaluation in Drug Discovery. Part 17: Development of

Recommend Documents