www.acsnano.org
Autonomous Scanning Probe Microscopy in Situ Tip Conditioning through Machine Learning Mohammad Rashidi*,†,‡ and Robert A. Wolkow†,‡,§ †
Department of Physics, University of Alberta, Edmonton, Alberta T6G 2J1, Canada Quantum Silicon, Inc., Edmonton, Alberta T6G 2M9, Canada § Nanotechnology Initiative, University of Alberta, Edmonton, Alberta T6G 2M9, Canada ‡
ABSTRACT: Atomic-scale characterization and manipulation with scanning probe microscopy rely upon the use of an atomically sharp probe. Here we present automated methods based on machine learning to automatically detect and recondition the quality of the probe of a scanning tunneling microscope. As a model system, we employ these techniques on the technologically relevant hydrogenterminated silicon surface, training the network to recognize abnormalities in the appearance of surface dangling bonds. Of the machine learning methods tested, a convolutional neural network yielded the greatest accuracy, achieving a positive identification of degraded tips in 97% of the test cases. By using multiple points of comparison and majority voting, the accuracy of the method is improved beyond 99%. KEYWORDS: scanning probe microscopy, machine learning, in situ tip conditioning, convolutional neural network, hydrogen terminated silicon, surface dangling bonds
T
and sample or controllably indenting the tip into the sample. These processes typically must be repeated many times before the tip’s quality is restored. Here, we use machine learning to automate in situ tip conditioning. Our process is based on a convolutional neural network (CNN) that is trained to analyze the quality of the tip. In our case, we work on the hydrogen terminated Si(100) substrate, a promising platform to develop atomic circuitry.12,20−24 The network is trained to recognize and assess the image quality of isolated surface dangling bonds (97% accuracy). By using majority voting on a small set of dangling bond images, the operational accuracy of the tip quality classification is improved beyond 99%. Upon detection of degraded probe quality, the routine performs in situ tip conditioning on a preselected spot on the surface. This procedure is repeated until the network registers a sharp probe.
he ability to directly visualize and manipulate individual atoms using scanning probe microscopy (SPM)1−11 has inspired scientists to develop atomic-scale technology for over two decades. Among other things, these technologies can be used to create smaller, more efficient, faster, and cheaper devices.12,13 To commercialize these technologies, SPM fabrication must become fast, precise, and autonomous. To this end, several studies have demonstrated methods that make it feasible to build parallelized atomically precise robots that manipulate and analyze atoms automatically.14−18 SPM techniques, and atomic manipulation in particular, rely on atomically sharp metal tips. While standard tip preparation methods can reliably produce them ex situ (for example, single atom tips can be prepared with field ion microscopy),19 during their use for imaging and atomic manipulation the quality of their apex is routinely compromised due to interactions with the surface. The signature of this is the loss of atomic resolution or the appearance of secondary imaging features, indicating that the apex of the tip no longer has a single predominant atom. Such tips are generally called “double tips” for this reason. Because a single atom tip is required for SPM atomic fabrication and experiments, in situ tip treatments are necessary to return the tip to its ideal (sharp) condition. This is usually the most time-consuming process for SPM operators. Common methods include applying short voltage pulses between the tip © XXXX American Chemical Society
RESULTS AND DISCUSSION For training we used approximately 3500 STM images (with train/test split ratio of 80/20) of isolated dangling bonds recorded at a sample bias of −1.8 V, where they typically appear as bright protrusions. These images were selected from five Received: March 23, 2018 Accepted: May 14, 2018
A
DOI: 10.1021/acsnano.8b02208 ACS Nano XXXX, XXX, XXX−XXX
Article
Cite This: ACS Nano XXXX, XXX, XXX−XXX
Article
ACS Nano years of archived data from two of our microscopes. To enable direct comparison, each 5.6 × 5.6 nm2 image was resized to 28 × 28 pixels. Each of the images was labeled manually (Figure 1). We augmented each image by rotating it by 90° four times
Table 1. Precision Score of Several Image Classification Models Evaluated on Our Test Dataseta model
PCC
KNN
RFC
SVM
FCNN
CNN
precision
0.77
0.84
0.89
0.88
0.78
0.97
a
PCC, KNN, RFC, SVM, FCNN and CNN denote Pearson’s correlation coefficient, K nearest neighbor, random forest classifier, support vector machine, fully connected neural network, and convolutional neural network, respectively. Of the KNN classifiers tested, the model with five neighbors resulted in the highest classification accuracy for our test dataset. 5000 trees were used for RFC. The (Gaussian) radial basis function kernel with C and γ parameters of 500 and 0.5 was used for SVM. The FCNN had 18 hidden layers (784 node each) with rectified-linear-unit (ReLU) activation function and Adam optimizer25 (learning rate of 10−3). The optimal values for the CNN are described in the main text.
Figure 1. Randomly selected labeled data set used for training. 5.6 × 5.6 nm2 dangling bond images are extracted from the STM images recorded at −1.8 V and 50 pA.
The white-dashed squares in Figure 3a indicate the dangling bond images that were used for analysis in this example. Figure 3c depicts the CNN architecture that yields the highest accuracy score on our test data set. It consists of two back to back 30 and 40 filters (5 × 5, stride 1) convolution layers followed by a max-pooling layer (2 × 2, stride 2) flattened and fully connected to a 128 node layer. The rectifiedlinear-unit (ReLU) activation function was used for the convolution and 128 node layers. A two-node densely connected layer with Softmax activation function was used as the output layer. The Adam algorithm25 with the learning rate of 10−4 and the categorical cross-entropy were employed for optimization and as the loss function, respectively. As an example, the output of each layer of the convolution layers is shown in Figure 3c for a dangling bond image in Figure 3b. The role of the convolution layers is to extract features from the input image using filters that are learned during the training process. The output images of each convolution layers, commonly called feature maps, result from sliding the corresponding filters over the input images of each layer. Different filters detect different features from an image. The pooling layer down-samples each feature maps while retaining the most important information. This reduces the computational cost during the training stage. The role of the densely connected layers is to learn nonlinear combinations of the feature maps and to classify the input image to various categories, for example, “sharp” and “double” tip. Figure 3d displays the output of the CNN for all the dangling bonds in the STM image. The program performs majority voting at the end to determine the outcome. We note that while we have used the characteristic circular appearance of isolated dangling bonds to train our neural network, these techniques can be applied generally to any material system with a recurrent surface feature (e.g., a welldefined surface reconstruction). These techniques are therefore directly accessible to groups that use SPM to study atomic adsorbates or defects,1−8 surface-supported molecules,26−28 or surfaces with recurrent features. Figure 4a shows an STM image obtained with a “double tip”. Our program successfully identified the tip was not ideal and performed tip conditioning in an attempt to restore the tip’s quality. Four conditioning steps were performed, at which point the program successfully recognized that the tip’s quality had been restored (Figure 4b) and the sequence was terminated. We note that the image frame to assess the quality of the tip must be carefully selected by the user to achieve accurate results. For instance, the defect close to the lower left dangling
and mirroring each rotated image. This expanded the training data set by 8 times and resulted in a significant performance increase to each of the models we tested. First, we used a method that does not rely upon machine learning to benchmark the performance of our image classification models. We chose Pearson’s correlation coefficient to determine the correlation of our data set with respect to a reference image taken with a “sharp tip” (Figure 2). For each
Figure 2. Correlation score of all images in the training data set with respect to a reference image taken with a “sharp tip”. The black dashed line indicates the threshold correlation score that yields the highest classification accuracy on the training data set.
reference image an optimal threshold correlation was computed by performing grid-search: Images with a correlation less than (greater than) the threshold were evaluated as “double tip” (“sharp tip”). The accuracy of the method was determined by counting the number of images which were misclassified using this optimal threshold. By using each “sharp tip” image in the training set as the reference image, we were able to select the reference image which yielded the highest accuracy on our test data set (77%). To determine the optimal image classification model, we compared the classification accuracy of the Pearson’s coefficient with several machine learning models, the results of which are summarized in Table 1. In our case a CNN yielded the highest accuracy (97% accuracy on the test data set) and was therefore used as the image classification model in our automated tip shaping routine. Figure 3 displays the workflow of the tip quality analysis using a CNN. Our routine automatically identifies and isolates subsections of the STM image containing dangling bonds and feeds them sequentially to the CNN to analyze the tip quality. B
DOI: 10.1021/acsnano.8b02208 ACS Nano XXXX, XXX, XXX−XXX
Article
ACS Nano
Figure 3. Tip quality analysis with convolutional neural network. (a) STM image (100 × 100 nm2) of hydrogen-terminated Si(100) recorded at −1.80 V and 50 pA. Bright features are surface dangling bonds. The dangling bonds are automatically extracted from the image (whitedashed squares) and sequentially fed into the CNN. (b) Close up of the dangling bond indicated by the red square in (a). (c) A depiction of the CNN used in this study. It consists of two convolution layers followed by a pooling layer, a densely connected layer, and an output layer. The result of the output layer is “0” for sharp tips or “1” for double tips. As an example, the output of each CNN layer is shown for the dangling bond image in (b). (d) The outputs of the CNN for each automatically extracted dangling bond image in (a).
note that because the tip must be placed precisely over the hydrogen to be desorbed, double tips are unsuitable for atomic fabrication because they result in high patterning error rates. After making the seventh atom, the quality of the tip unexpectedly deteriorated, and the user initiated the tipconditioning routine to fix it. The routine successfully detected the double tip by assessing the STM image of the isolated DB at middle left of Figure 5a (assessments shown in Figure 5e,f) and performed tip conditioning at the preselected spot. After three tip conditioning events, the routine correctly identified that the tip quality had been restored, and the user resumed fabrication. Figure 5d displays the final wire with eight atoms, where we note the superb image quality.
Figure 4. An example of automatic tip sharpening. (a) Initial image to judge the quality of the tip. User sets an image frame and a spot for in situ tip conditioning before starting the program. The majority vote output of the CNN is “1” (double tip) for this image and 4 other subsequent images (not shown here), indicating that the tip conditioning was not successful. (b) After a successful tip conditioning, the majority vote output of the CNN becomes “0” (sharp tip) and the program stops its operation. The size of the images were 40 × 40 nm2, and the tunneling conditions were −1.8 V and 50 pA.
CONCLUSIONS As a starting point for developing parallel atomic-precision fabrication tools, we have implemented an automated routine that can evaluate the quality of an SPM probe and perform in situ conditioning to restore the quality of degraded tips. This routine is based on a CNN that assesses the probe quality by evaluating images of known atomic defects. As an example, we trained the CNN with images of silicon dangling bonds and demonstrated its use during the patterning of a binary atomic wire. The framework that we have developed here is an important step toward creating autonomous atom-scale fabrication tools and will be added as a module to the fully automated SPM patterning program we are currently developing. This framework could be applied to other material systems and nanoscale imaging techniques. Applications other than atom-scale fabrication, such as critical dimension analysis as used in modern semiconductor fabrication, will also benefit by variants of the techniques described here.
bond in Figure 4b results in the outcome of “double tip” for that dangling bond. Because all the other dangling bonds in the image frame are isolated, the program detects an accurate outcome. As an example of how these techniques can be interfaced directly with existing autonomous atomic fabrication routines, we demonstrate the use of our tip conditioning routine during the fabrication of a binary atomic wire from silicon dangling bonds.12 Figure 5a displays an STM image of the area that the experiment is performed. The user chose a defect-free area to fabricate the binary wire, an isolated DB to assess the tip quality using the CNN, and a spot to perform tip indentation to recondition the tip. The wire was fabricated one atom at a time (Figure 5b) by desorbing hydrogen atoms using voltage pulses (pulsed to 2.10 V at the set-point of +1.3 V and 50 pA). We
METHODS All experiments were performed on an Omicron LT STM operating at 4.5 K and under ultrahigh vacuum. The tips were electrochemically etched from polycrystalline tungsten wire. Tips were heated via C
DOI: 10.1021/acsnano.8b02208 ACS Nano XXXX, XXX, XXX−XXX
Article
ACS Nano
Figure 5. Autonomous tip sharpening used along with atom-scale patterning. (a) An overview (25 × 25 nm2) STM image showing a patterned binary atomic wire, an isolated DB used for tip quality assessment, and a spot (red cross) chosen by the user to perform tip conditioning. (b) Sequence of patterning steps without noticeable tip quality change in between. (c) The tip became double after the creation of the last atom on the right, and the user employed the tip conditioning routine to resharpen it. After three steps of automatic tip conditioning, the tip became sharp and the user carried on the pattering (d). (e) STM image of the isolated DB at middle left of (a) after each tip conditioning. The CNN used these images to assess the quality of the tip. (f) Output of the CNN for the images in (e). The tunneling conditions were −1.8 V and 50 pA for all images. electron bombardment in ultrahigh vacuum to remove the surface oxide and sharpened to single atom by field ion microscopy.19 In situ tip processing was performed by controlled tip indentation with the surface. This procedure was shown in prior studies10,29,30 to give rise to silicon tips. Before initiating the tip-conditioning routine, a bias voltage, a tip approach speed, and a range of tip approach distances were set for the tip indentation process. For the examples shown in the manuscript, the tip approach distance range was 700 pm to 1 nm from the set-point of −1.8 V and 50 pA, the bias voltage during indentation was −1.8 V, and the tip approach speed was approximately 10−8 m/s. After each unsuccessful tip conditioning attempt, the tip approach distance was incremented by 10 pm. We note that other common in situ tip conditioning methods, for example, metal dipping or picking up molecules, could readily be incorporated into the procedures outlined here. Samples are highly arsenic-doped (1.5 × 1019 atom/cm3) Si(100). Samples were degassed at 600 °C for 12 h followed by flash annealing at 1250 °C. For hydrogen passivation, they are exposed to atomic hydrogen gas at 330 °C. A Nanonis SPM controller was used for imaging and data acquisition. The tip conditioning automation routine was programmed in Python and Labview using the Nanonis programming interface. A knearest neighbor, random forest, support vector machine, and fully connected neural network were implemented using the ScikitLearn(0.19.1), Python machine learning library. The CNN was implemented using Keras(2.1.3) with TensorFlow backend.
Compute Canada, and NSERC for their support. M.R. thanks Wyatt Vine and Ken Gordon for their help to edit and proofread the manuscript.
REFERENCES (1) Eigler, E. K.; Schweizer, D. K. Positioning Single Atoms with a Scanning Tunneling Microscop. Nature 1990, 344, 524−526. (2) Crommie, M. F.; Lutz, C. P.; Eigler, D. M. Confinement of Electrons to Quantum Corrals on a Metal Surface. Science 1993, 262, 218−220. (3) Sugimoto, Y.; Pou, P.; Custance, O.; Jelinek, P.; Abe, M.; Perez, R.; Morita, S. Complex Patterning by Vertical Interchange Atom Manipulation Using Atomic Force Microscopy. Science 2008, 322, 413−417. (4) Kalff, F. E.; Rebergen, M. P.; Fahrenfort, E.; Girovsky, J.; Toskovic, R.; Lado, J. L.; Fernández-Rossier, J.; Otte, A. F. A Kilobyte Rewritable Atomic Memory. Nat. Nanotechnol. 2016, 11, 926−929. (5) Slot, M. R.; Gardenier, T. S.; Jacobse, P. H.; van Miert, G. C. P.; Kempkes, S. N.; Zevenhuizen, S. J. M.; Smith, C. M.; Vanmaekelbergh, D.; Swart, I. Experimental Realization and Characterization of an Electronic Lieb lattice. Nat. Phys. 2017, 13, 672−676. (6) Drost, R.; Ojanen, T.; Harju, A.; Liljeroth, P. Topological States in Engineered Atomic Lattices. Nat. Phys. 2017, 13, 668−671. (7) Fölsch, S.; Martínez-Blanco, J.; Yang, J.; Kanisawa, K.; Erwin, S. C. Quantum Dots with Single-Atom Precision. Nat. Nanotechnol. 2014, 9, 505−508. (8) Kawai, S.; Foster, A. S.; Canova, F. F.; Onodera, H.; Kitamura, S.i.; Meyer, E. Atom Manipulation on an Insulating Surface at Room Temperature. Nat. Commun. 2014, 5, 4403. (9) Stroscio, J. A.; Celotta, R. J. Controlling the Dynamics of a Single Atom in Lateral Atom Manipulation. Science 2004, 306, 242−248. (10) Huff, T. R.; Labidi, H.; Rashidi, M.; Koleini, M.; Achal, R.; Salomons, M. H.; Wolkow, R. A. Atomic White-Out: Enabling Atomic Circuitry through Mechanically Induced Bonding of Single Hydrogen Atoms to a Silicon Surface. ACS Nano 2017, 11, 8636−8642. (11) Pavliček, N.; Majzik, Z.; Meyer, G.; Gross, L. Tip-Induced Passivation of Dangling Bonds on Hydrogenated Si(100)-2 × 1. Appl. Phys. Lett. 2017, 111, 053104. (12) Huff, T.; Labidi, H.; Rashidi, M.; Achal, R.; Livadaru, L.; Dienel, T.; Pitters, J.; Wolkow, R. A. Binary Atomic Silicon Logic. arXiv:1706.07427 2017.
AUTHOR INFORMATION Corresponding Author
*E-mail:
[email protected]. ORCID
Mohammad Rashidi: 0000-0002-8830-4812 Notes
The authors declare the following competing financial interest(s): A patent has been filed on the subject and both authors are affiliated with Quantum Silicon Inc.
ACKNOWLEDGMENTS We would like to thank Mark Salomons and Martin Cloutier for their technical expertise. We also thank Alberta Innovates, D
DOI: 10.1021/acsnano.8b02208 ACS Nano XXXX, XXX, XXX−XXX
Article
ACS Nano (13) Khajetoorians, A. A.; Wiebe, J.; Chilian, B.; Wiesendanger, R. Realizing All-Spin-Based Logic Operations Atom by Atom. Science 2011, 332, 1062−1064. (14) Woolley, R. A. J.; Stirling, J.; Radocea, A.; Krasnogor, N.; Moriarty, P. Automated Probe Microscopy via Evolutionary Optimization at the Atomic Scale. Appl. Phys. Lett. 2011, 98, 253104. (15) Stirling, J.; Woolley, R. A. J.; Moriarty, P. Scanning Probe Image Wizard: A Toolbox for Automated Scanning Probe Microscopy Data Analysis. Rev. Sci. Instrum. 2013, 84, 113701. (16) Møller, M.; Jarvis, S. P.; Guérinet, L.; Sharp, P.; Woolley, R.; Rahe, P.; Moriarty, P. Automated Extraction of Single H Atoms with STM: Tip State Dependency. Nanotechnology 2017, 28, 075302. (17) Ziatdinov, M.; Maksov, A.; Kalinin, S. V. Learning Surface Molecular Structures via Machine Vision. npj Comput. Mater. 2017, 3, 31. (18) Ziatdinov, M.; Dyck, O.; Maksov, A.; Li, X.; Sang, X.; Xiao, K.; Unocic, R. R.; Vasudevan, R.; Jesse, S.; Kalinin, S. V. Deep Learning of Atomically Resolved Scanning Transmission Electron Microscopy Images: Chemical Identification and Tracking Local Transformations. ACS Nano 2017, 11, 12742−12752. (19) Rezeq, M.; Pitters, J.; Wolkow, R. Tungsten Nanotip Fabrication by Spatially Controlled Field-Assisted Reaction with Nitrogen. J. Chem. Phys. 2006, 124, 204716. (20) Livadaru, L.; Xue, P.; Shaterzadeh-Yazdi, Z.; DiLabio, G. a.; Mutus, J.; Pitters, J. L.; Sanders, B. C.; Wolkow, R. a. Dangling-Bond Charge Qubit on a Silicon Surface. New J. Phys. 2010, 12, 083018. (21) Rashidi, M.; Taucer, M.; Ozfidan, I.; Lloyd, E.; Koleini, M.; Labidi, H.; et al. Time-Resolved Imaging of Negative Differential Resistance on the Atomic Scale. Phys. Rev. Lett. 2016, 117, 276805. (22) Schofield, S. R.; Studer, P.; Hirjibehedin, C. F.; Curson, N. J.; Aeppli, G.; Bowler, D. R. Quantum Engineering at the Silicon Surface Using Dangling Bonds. Nat. Commun. 2013, 4, 1649. (23) Fuechsle, M.; Miwa, J. A.; Mahapatra, S.; Ryu, H.; Lee, S.; Warschkow, O.; Hollenberg, L. C. L.; Klimeck, G.; Simmons, M. Y. A Single-Atom Transistor. Nat. Nanotechnol. 2012, 7, 242−246. (24) Rashidi, M.; Lloyd, E.; Huff, T. R.; Achal, R.; Taucer, M.; Croshaw, J. J.; Wolkow, R. A. Resolving and Tuning Carrier Capture Rates at a Single Silicon Atom Gap State. ACS Nano 2017, 11, 11732− 11738. (25) Kingma, D. P.; Ba, J. L. ADAM: A Method for Stochastic Optimization. arXiv:1412.6980 2015. (26) Gross, L.; Mohn, F.; Moll, N.; Liljeroth, P.; Meyer, G. The Chemical Structure of a Molecule Resolved by Atomic Force Microscopy. Science 2009, 325, 1110−1114. (27) Rashidi, M.; Müllegger, S.; Roithner, M.; Schöfberger, W.; Koch, R. Spectroscopic Scanning Tunneling Microscopy Studies of Single Surface-Supported Free-Base Corroles. J. Am. Chem. Soc. 2012, 134, 91−94. (28) Müllegger, S.; Rashidi, M.; Lengauer, T.; Rauls, E.; Schmidt, W. G.; Knör, G.; Schöfberger, W.; Koch, R. Asymmetric Saddling of Single Porphyrin Molecules on Au(111). Phys. Rev. B: Condens. Matter Mater. Phys. 2011, 83, 165416. (29) Jarvis, S.; Sweetman, A.; Bamidele, J.; Kantorovich, L.; Moriarty, P. Role of Orbital Overlap in Atomic Manipulation. Phys. Rev. B: Condens. Matter Mater. Phys. 2012, 85, 235305. (30) Labidi, H.; Koleini, M.; Huff, T.; Salomons, M.; Cloutier, M.; Pitters, J.; Wolkow, R. A. Indications of Chemical Bond Contrast in AFM Images of a Hydrogen-Terminated Silicon Surface. Nat. Commun. 2017, 8, 14222.
E
DOI: 10.1021/acsnano.8b02208 ACS Nano XXXX, XXX, XXX−XXX
