Augmented Reality in Scientific Publications ... - ACS Publications

In Focus Cite This: ACS Chem. Biol. 2018, 13, 496−499

Augmented Reality in Scientific PublicationsTaking the Visualization of 3D Structures to the Next Level Patrik Wolle, Matthias P. Müller, and Daniel Rauh* Faculty of Chemistry and Chemical Biology, TU Dortmund University, Otto-Hahn-Strasse 4a, D-44227 Dortmund, Germany ABSTRACT: The examination of three-dimensional structural models in scientific publications allows the reader to validate or invalidate conclusions drawn by the authors. However, either due to a (temporary) lack of access to proper visualization software or a lack of proficiency, this information is not necessarily available to every reader. As the digital revolution is quickly progressing, technologies have become widely available that overcome the limitations and offer to all the opportunity to appreciate models not only in 2D, but also in 3D. Additionally, mobile devices such as smartphones and tablets allow access to this information almost anywhere, at any time. Since access to such information has only recently become standard practice, we want to outline straightforward ways to incorporate 3D models in augmented reality into scientific publications, books, posters, and presentations and suggest that this should become general practice. figure even during yet another tedious evening meeting, or on the way home in the subway. In this publication, we present figures of small organic molecules, protein−ligand interactions, and larger protein complexes supplemented with QR codes that allow the reader to appreciate the complexity of a structure also in 3D, thus offering the possibility to rotate the depicted molecule to a different perspective or zoom into the active site. Figure 1 shows some examples of small organic molecules bound to protein targets and highlights key interactions. Amino acids that are not well visualized in 2D (e.g., Met790 and Lys745 in Figure 1A) can be easily rotated to the foreground in AR, and the electron density surrounding the small molecule can be viewed from different angles. Similarly, the geometry of interactions (e.g., the interactions between the hinge region and the inhibitor in Figure 1A or the π−π-stacking interaction between adenosine-5′-monophosphate and Phe45 in Figure 1C) or interaction surfaces (Figure 1D) can be inspected. However, not only protein structures can be visualized in AR but also small molecules. As an example, we prepared the structure of morphine visualized as Ortep (Oak Ridge Thermal Ellipsoid) plot (Figure 2). The structure illustrates the complex stereochemistry, which is difficult to visualize in a 2D figure, but can be easily described in AR for, e.g., teaching purposes. Finally, in Figure 3 we want to highlight the possibility to also visualize large protein molecules in 3D, including the possibility for the reader to zoom into an area of interest, e.g., the pore of a large toxin (Figure 3A) or the active site of RNA-polymerase III (Figure 3B) showing the mechanism of action of the corresponding protein.

T

he elucidation of 3D structures of molecules has revolutionized our understanding of how structure is tightly connected to function, biology, and life. Methods to solve 3D structures as well as innovative programs such as PyMOL, UCSF Chimera, and others have enabled us to visualize, interpret, and actively interact with complex structural data. This is best represented by the ever-increasing number of scientific publications describing 3D models of proteins and/or small molecules obtained via X-ray crystallography, NMR, or EM. Even though the models are usually publicly deposited,1,2 the original publications describing them naturally only contain two-dimensional figures prepared in a certain orientation to outline whatever message the author(s) wants to convey. However, to properly examine the conclusions, one usually needs to reproduce the corresponding figures in 3D in an appropriate software package3−6 (even though other solutions exist, e.g., 3D interactive content embedded in a PDF7). Depending on the complexity of the figure, this approach demands a great amount of time and knowledge that are both not necessarily available to every reader. Stimulated by the recent advances and developments in augmented and virtual reality (AR and VR, respectively), and the increased power of GPUs (graphics processing units) available in regular mobile devices such as smartphones and tablets, we suggest that these can and should be implemented into scientific research. While the use of AR and VR to visualize (e.g., for teaching purposes)8 or even to manipulate threedimensional structures9 is not a novel approach, it has not been widely used in the life sciences. In this publication, we will present a simple procedure on how this can be done and present a few examples of 2D figures and the corresponding 3D AR figures that permit the reader to quickly obtain a better stereoscopic view, allowing access to the three-dimensional © 2018 American Chemical Society

Published: March 16, 2018 496

DOI: 10.1021/acschembio.8b00153 ACS Chem. Biol. 2018, 13, 496−499

In Focus

ACS Chemical Biology

Figure 1. Examples of three-dimensional structural representations of protein molecules. (A) The active site of the epidermal growth factor receptor (EGFR) kinase domain bound covalently via Cys797 to a pyrazolopyrimidine inhibitor (PDB ID 5j9y; 2FO−FC map contoured at 1σ).12 (B) Small molecule inhibitor bound to the lipophilic binding pocket of the kinase p38α. Surrounding secondary structure elements (αEF/αF loop and helices 1L14 and 2L14) and amino acids (Trp197, Lys249, Ser251, and Asp294) involved in binding are shown. The 2FO−FC electron density is depicted as a mesh at 1σ (PDB ID 5n67).13 (C) The small GTPase Rab1b bound to guanosine-5′-[β,γimido]-triphosphate (GppNHp) and Mg2+ (green sphere) in the active site and covalently modified with adenosine-5′-monophosphate at Tyr77 (the important switch I (red), switch II (blue), and P-loop (magenta) are highlighted; PDB ID 3nkv).14 (D) Model of the transcription factor Nrf2 (cartoon) bound to the major groove of a short stretch of DNA (surface representation) highlighting important basic residues that are presumably involved in recognition of the nucleic acid and interaction with the negatively charged phosphate groups of the DNA (the model is based on PDB ID 1skn).

Figure 3. Large molecular assemblies can also be visualized in augmented reality. (A) Cartoon representation of the pentameric toxin TcdA1 (PDB ID 4o9y) in two different orientations (the monomers are shown in orange, purple, gray, turquoise, and green and consist of ∼2500 amino acids each). Note the long channel within the pentamer of the protein that is used to funnel the toxin into infected cells.16 (B) The RNA-polymerase III open preinitiation complex (PDB ID 6eu0). The structure was obtained by cryoelectron microscopy and shows the polymerase bound to the partially unwound DNA and Mg2+ in the active site.17 processing software such as MeshLab (http://www.meshlab.net/). This program can be used to edit the polygon meshes in various ways and is needed to reduce the number of faces (and thus the file size. Augment.com has a file size limit of 100 MB, and you must keep in mind that especially older smartphones have limitations with visualization of bigger structures containing large amounts of information). A simple tool to reduce the number of faces is the “Simplification: Quadric Edge Collapse Decimation” filter in MeshLab (Filters → Remeshing, Simplification and Reconstruction → Simplification: Quadric Edge Collapse Decimation). Depending on the size and complexity of the 3D structural model, one needs to determine how much the number of faces can be reduced without losing too much information. Additionally, to ensure correct display of colors in various programs, one needs to copy the colors from the vertices to the faces (Filters → Color Creation and Processing → Transfer Color: Vertex to Face). The 3D figure can then be exported again in the Wavefront file format (.obj) and Blender is used to convert the file format to COLLADA (.dae) format that can be uploaded directly to Augment.com to obtain QR codes. If a reduction of the file size is not necessary, VRML2-files can also be directly converted to COLLADA format in blender (Figure 4). Viewing the Structures with Augment. Install the free Augment app from the app store or directly from http://www. augment.com/augmented-reality-apps/. With Apple devices simply point the camera on the QR code. With Android devices scan the QR code with any QR-code scanner or the one embedded in the augment app. The 3D model will be displayed in “live view” in the real environment (e.g., on your desktop) using the camera of the mobile device. Alternatively, the display can be switched to 3D view to display the model without any background environment. In both views, one can navigate and zoom into different parts of the structure using the touch screen of the mobile device.

Figure 2. Simple visualization of complex stereochemistry. The small molecule X-ray structure of morphine including five asymmetric carbon atoms (*) shown as a two-dimensional figure (A) and the corresponding Ortep plot (CCDC identifier EFASAH15). (B) The view in 3D AR allows the reader to directly appreciate the complex stereochemistry of this molecule (carbon, gray; oxygen, red; nitrogen, blue; Cl−, green).

■

EXPERIMENTAL SECTION

Generating 3D Structural Models for Augment. To generate 3D structural models for Augment, corresponding 3D figures must first be prepared with a molecular structure viewer like PyMOL3 or UCSF Chimera.6 Figures can either be exported directly in the COLLAborative Design Activity (COLLADA) format (.dae) for simple figures, or a slightly more advanced procedure can be used for more complex cases (e.g., including electron density) as outlined below. To do this, one needs to export the 3D figure in the Virtual Reality Modeling Language (VRML2) format and import it into 3D computer graphics software like Blender (https://www.blender.org/). In this step, Blender is only used to change the format to Wavefront (.obj). This file is then imported into 3D mesh converting and 497

DOI: 10.1021/acschembio.8b00153 ACS Chem. Biol. 2018, 13, 496−499

In Focus

ACS Chemical Biology

Besides the presented application (AUGMENT, Augment USA, Inc.), other visualization software (e.g., ChemPreview11) and procedures are available to produce and visualize threedimensional representations in virtual and augmented reality. This publication is not meant to give a comprehensive overview, but to introduce the possibility to a broader audience and to provide simple instructions on how this can be done.

■

AUTHOR INFORMATION

Corresponding Author

*Phone: +49 (0)231−755 7080. Fax: +49 (0)231−755 7082. E-mail: [email protected]. ORCID

Matthias P. Müller: 0000-0002-1529-8933 Daniel Rauh: 0000-0002-1970-7642

■

ACKNOWLEDGMENTS This work was cofunded by the German Federal Ministry for Education and Research (NGFNPlus and e:Med; Grant No. BMBF 01GS08104, 01ZX1303C), the Deutsche Forschungsgemeinschaft (DFG), the German federal state North RhineWestphalia (NRW), the European Union (European Regional Development Fund: Investing In Your Future; EFRE-800400), NEGECA (PerMed NRW), and EMODI. We are very thankful to all the people involved in the open source software projects blender (https://www.blender.org/) and MeshLab (http:// www.meshlab.net/) as well as the team behind the Augment app.

Figure 4. Schematic representation of the workflow for generating 3D figures for Augment. Simple figures can be directly exported from PyMOL in the VRML2 format and converted in Blender to the COLLADA format (left). For more complex figures (e.g., electron density maps, surface representations, and hydrogen bonding interactions), an additional step is needed to decrease the complexity (further information can be found in the main text). The COLLADA file can be uploaded to Augment.com to obtain a QR code that can be embedded in the publication.

■

REFERENCES

(1) Berman, H. M., Westbrook, J., Feng, Z., Gilliland, G., Bhat, T. N., Weissig, H., Shindyalov, I. N., and Bourne, P. E. (2000) The Protein Data Bank. Nucleic Acids Res. 28, 235−242. (2) Groom, C. R., Bruno, I. J., Lightfoot, M. P., and Ward, S. C. (2016) The Cambridge Structural Database,. Acta Crystallogr., Sect. B: Struct. Sci., Cryst. Eng. Mater. 72, 171−179. (3) DeLano, W. L., and Lam, J. W. (2005) PyMOL: A communications tool for computational models. Abstr. Pap.Am. Chem. Soc. 230, U1371−U1372. (4) Emsley, P., Lohkamp, B., Scott, W. G., and Cowtan, K. (2010) Features and development of Coot. Acta Crystallogr., Sect. D: Biol. Crystallogr. 66, 486−501. (5) McNicholas, S., Potterton, E., Wilson, K. S., and Noble, M. E. M. (2011) Presenting your structures: the CCP4mg molecular-graphics software. Acta Crystallogr., Sect. D: Biol. Crystallogr. 67, 386−394. (6) Pettersen, E. F., Goddard, T. D., Huang, C. C., Couch, G. S., Greenblatt, D. M., Meng, E. C., and Ferrin, T. E. (2004) UCSF chimera - A visualization system for exploratory research and analysis. J. Comput. Chem. 25, 1605−1612. (7) Kumar, P., Ziegler, A., Grahn, A., Hee, C. S., and Ziegler, A. (2010) Leaving the structural ivory tower, assisted by interactive 3D PDF. Trends Biochem. Sci. 35, 419−422. (8) Berry, C., and Board, J. (2014) A Protein in the Palm of Your Hand Through Augmented Reality,. Biochem. Mol. Biol. Educ. 42, 446− 449. (9) Anderson, A., and Weng, Z. P. (1999) VRDD: Applying virtual reality visualization to protein docking and design. J. Mol. Graphics Modell. 17, 180. (10) Olshannikova, E., Ometov, A., Koucheryavy, Y., and Olsson, T. (2015) Visualizing Big Data with augmented and virtual reality: challenges and research agenda. Journal of Big Data 2, 22. (11) Zheng, M., and Waller, M. P. (2017) ChemPreview: an augmented reality-based molecular interface. J. Mol. Graphics Modell. 73, 18−23.

To experience an even more interactive session with the structure, you can define a “tracker.” Click on the tracker icon and follow the instructions. The tracker works best when using a detailed object, e.g., paper with written text or a colorful business card. You can now turn the tracker, and the visualized structure will follow (see http://www. augment.com/trackers/).

■

CONCLUSION The use of software and tools described in this publication can by no means replace sophisticated visualization software of three-dimensional protein models such as PyMOL, 3 CCP4MG,5 UCSF Chimera,6 or Coot4 that can be used to manipulate and visualize a model in almost infinite ways. However, the procedure we describe allows quick access to information that is not simply available from a 2D figure. Since the production of corresponding figures supplemented with AR is straightforward, we suggest that it should become standard practice to be incorporated (e.g., by supplying QR codes) into figures representing structural data. Similarly, AR can be used in lectures or seminars in the class room or on scientific conferences in poster sessions and oral presentations by projecting/presenting the QR code, and the audience can directly appreciate the structure on their mobile devices. Even though it may be difficult to refocus the audiences’ attention for the remaining duration of the lecture, we think that it can be a beneficial addition that should be more widely used. Applying this strategy is certainly not limited to structural models of proteins and other molecules, but could be generally used for other objects such as cells or for elaborate three-dimensional data sets as found from the results from mass spectrometry experiments to map interaction partners.10 498

DOI: 10.1021/acschembio.8b00153 ACS Chem. Biol. 2018, 13, 496−499

In Focus

ACS Chemical Biology (12) Engel, J., Becker, C., Lategahn, J., Keul, M., Ketzer, J., Muhlenberg, T., Kollipara, L., Schultz-Fademrecht, C., Zahedi, R. P., Bauer, S., and Rauh, D. (2016) Insight into the Inhibition of DrugResistant Mutants of the Receptor Tyrosine Kinase EGFR. Angew. Chem., Int. Ed. 55, 10909−10912. (13) Bührmann, M., Wiedemann, B. M., Müller, M. P., Hardick, J., Ecke, M., and Rauh, D. (2017) Structure-based design, synthesis and crystallization of 2-arylquinazolines as lipid pocket ligands of p38alpha MAPK. PLoS One 12, e0184627. (14) Müller, M. P., Peters, H., Blumer, J., Blankenfeldt, W., Goody, R. S., and Itzen, A. (2010) The Legionella effector protein DrrA AMPylates the membrane traffic regulator Rab1b,. Science 329, 946− 949. (15) Gelbrich, T., Braun, D. E., and Griesser, U. J. (2012) Morphine hydro-chloride anhydrate. Acta Crystallogr., Sect. E: Struct. Rep. Online 68, o3358−3359. (16) Meusch, D., Gatsogiannis, C., Efremov, R. G., Lang, A. E., Hofnagel, O., Vetter, I. R., Aktories, K., and Raunser, S. (2014) Mechanism of Tc toxin action revealed in molecular detail. Nature 508, 61. (17) Abascal-Palacios, G., Ramsay, E. P., Beuron, F., Morris, E., and Vannini, A. (2018) Structural basis of RNA polymerase III transcription initiation. Nature 553, 301−306.

499

DOI: 10.1021/acschembio.8b00153 ACS Chem. Biol. 2018, 13, 496−499