Editorial: Tox21 Challenge to Build Predictive Models of Nuclear Receptor and Stress Response Pathways As Mediated by Exposure to Environmental Toxicants and Drugs

Editorial on the Research Topic
Tox21 Challenge to Build Predictive Models of Nuclear Receptor and Stress Response Pathways As Mediated by Exposure to Environmental Toxicants and Drugs

Tens of thousands of chemicals are released into the environment every year. High-throughput screening (HTS) has offered a more efficient and cost-effective alternative to traditional toxicity tests to profile these chemicals for potential adverse effects with the aim to prioritize a manageable number for in depth testing and to provide clues to their mechanisms of toxicity. The Tox21 program (NRC, 2007; Collins et al., 2008; Kavlock et al., 2009; Tice et al., 2013), a collaboration among the National Institute of Environmental Health Sciences (NIEHS)/National Toxicology Program (NTP), the U.S. Environmental Protection Agency's (EPA) National Center for Computational Toxicology (NCCT), the National Institutes of Health (NIH) National Center for Advancing Translational Sciences (NCATS), and the U.S. Food and Drug Administration (FDA), has generated quantitative high-throughput screening (qHTS) data (>50 million data points) on a library of 10K compounds, including environmental chemicals and drugs, against a panel of nuclear receptor and stress response pathway assays during its production phase (phase II) (Huang et al., 2016). A worldwide modeling competition, the Tox21 Data Challenge, was launched that asked a crowd of researchers to use these data as the training set to elucidate the extent to which the interference of biochemical and cellular pathways by compounds can be inferred from chemical structure data. This E-book comprises articles describing computational models with good predictive performance that resulted from this challenge (Huang et al.).

Machine learning methods have been widely used by the computational modeling community for the prediction of biological activity induced by small molecules. The Tox21 Data Challenge provides the unique opportunity to compare the predictive abilities of different computational methods for biological activity, specifically, those related to toxicity. The Challenge participants employed a wide range of chemical descriptors and/or fingerprints for small molecule representation, and machine learning algorithms for modeling.

Models employed the deep learning algorithm showed the best predictive performance. Deep Learning, as a field of machine learning, has gained popularity in the recent years, having been widely applied in the fields such as signal and information processing, speech recognition, as well as physics and life sciences. Deep Learning has also been applied to predict the outcome of biological assays. Mayr and coauthors, were the first to apply Deep Learning to computational toxicity (Mayr et al.). Deep Learning enabled them to construct a hierarchy of chemical features that combines the best of the features to ensembles, resulting in models that outperformed most other computational methods. Other than the high performance but computationally intensive Deep Learning method, simple traditional machine learning methods also attained success in predicting a number of assay activities. Abdelaziz and coauthors developed consensus models with methods implemented within the OCHEM (http://www.ochem.eu) web-based platform using 10 different descriptor sets and the associative neural networks (ASNN) algorithm (Abdelaziz et al.). These consensus models achieved the best overall balanced accuracy across all assays and top performance in the ATAD5 and mitochondrial membrane potential disruption assays. Their stratified bagging contributed models, and the selection of consensus models, were optimized to achieve the best balanced accuracy. Barta employed the ensemble approach for model development, combining various fingerprinting tools with different machine learning techniques, and applied assorted feature selection methods (Barta). Barta found that multi-tree ensemble methods, such as Random Forests and Extra Trees, produce reliable predictions and are insensitive to dimensionality extremes. These models achieved the best performance in predicting compound activities against AR, aromatase, and p53. Random Forest was also the method of choice for Uesawa, who produced the best performing ER-LBD model, calculated multiple descriptors and applied Random Forest for descriptor selection and model generation (Uesawa).

Other articles described models that employed classic machine learning algorithms, such as Random Forest, Support Vector Machine (SVM), k Nearest Neighbor (kNN), and Naïve Bayes (Drwal et al.), with different combinations of molecular descriptors, each with their own spin on the specific implementation of these methods for model construction. Capuzzi and coauthors found that their models built with Deep Neural Networks performed better than those developed with simple machine learning algorithms and that dataset balancing had a detrimental effect on prediction accuracy (Capuzzi et al.). Stefaniak evaluated combinations of various attribute selection methods and machine learning algorithms and determined that combining the Best First method for attribute selection with the Rotation Forest/ADTree classifier produced the best models (Stefaniak). Koutsoukas and coauthors utilized circular molecular fingerprints combined with Random Forest and SVM (Koutsoukas et al.). Ribay and coauthors also applied the biological response profile of chemicals from public data sources toward model construction, and found significant improvement in model performance compared to models built with chemical structure information alone (Ribay et al.).

Other than the Deep Learning techniques, there is no clear indication of which machine learning algorithm and/or molecular descriptor, or a specific combination of the two, has significant advantage over the others. Specific implementation of the methods and application of the optimal methods to the most fitting dataset seem to make the most difference. Common strategies employed by the best performing models show that consensus modeling and the diversity of descriptors tend to improve the predictive performance of models. The top performing models reached prediction accuracies close to the level of experimental errors (Huang et al.), demonstrating the feasibility of applying these models as screening tools for chemical prioritization.

The articles from this e-book present a groundbreaking direction for toxicological related testing and are intended to help improve the understanding of how chemicals could disrupt biological pathways and result in toxicity. Specifically, the computational models generated from this Challenge can be applied to predict the potential of those environmental chemicals with limited information to disrupt nuclear receptor and cellular stress response pathways. The computational models built within this Challenge are expected to improve the community's ability to prioritize novel chemicals with respect to potential concern to human health. The best performing models are currently being made publicly accessible to the scientific community (Abdelaziz et al.; Mayr et al.) to help facilitate chemical risk assessment.

Author Contributions

RH and MX wrote the editorial.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This work was supported by the Intramural Research Programs of the National Toxicology Program (Interagency agreement #Y2-ES-7020-01), National Institute of Environmental Health Sciences, the U.S. Environmental Protection Agency (Interagency Agreement #Y3-HG-7026-03), and the National Center for Advancing Translational Sciences, National Institutes of Health. We would like to take this opportunity to thank all the authors for their excellent contributions that made this series a success, and the reviewers for their insightful comments and remarks.

References

Collins, F. S., Gray, G. M., and Bucher, J. R. (2008). Toxicology. Transforming environmental health protection. Science 319, 906–907. doi: 10.1126/science.1154619

PubMed Abstract | CrossRef Full Text | Google Scholar

Huang, R., Xia, M., Sakamuru, S., Zhao, J., Shahane, S. A., Attene-Ramos, M., et al. (2016). Modelling the Tox21 10 K chemical profiles for in vivo toxicity prediction and mechanism characterization. Nat. Commun. 7:10425. doi: 10.1038/ncomms10425

PubMed Abstract | CrossRef Full Text | Google Scholar

Kavlock, R. J., Austin, C. P., and Tice, R. R. (2009). Toxicity testing in the 21st century: implications for human health risk assessment. Risk Anal. 29, 485–487. discussion: 492–487. doi: 10.1111/j.1539-6924.2008.01168.x

PubMed Abstract | CrossRef Full Text | Google Scholar

NRC (2007). Toxicity Testing in the 21st Century: A Vision and a Strategy. Washington, DC: The National Academies Press.

Tice, R. R., Austin, C. P., Kavlock, R. J., and Bucher, J. R. (2013). Improving the human hazard characterization of chemicals: a Tox21 update. Environ. Health Perspect. 121, 756–765. doi: 10.1289/ehp.1205784

PubMed Abstract | CrossRef Full Text | Google Scholar