Background Estimation and Correction for High-Precision Localization

Subscriber access provided by Binghamton University | Libraries

Article

Background Estimation and Correction for High-Precision Localization Microscopy Ching-Ya Cheng, and Chia-Lung Hsieh ACS Photonics, Just Accepted Manuscript • Publication Date (Web): 08 Jun 2017 Downloaded from http://pubs.acs.org on June 11, 2017

Just Accepted “Just Accepted” manuscripts have been peer-reviewed and accepted for publication. They are posted online prior to technical editing, formatting for publication and author proofing. The American Chemical Society provides “Just Accepted” as a free service to the research community to expedite the dissemination of scientific material as soon as possible after acceptance. “Just Accepted” manuscripts appear in full in PDF format accompanied by an HTML abstract. “Just Accepted” manuscripts have been fully peer reviewed, but should not be considered the official version of record. They are accessible to all readers and citable by the Digital Object Identifier (DOI®). “Just Accepted” is an optional service offered to authors. Therefore, the “Just Accepted” Web site may not include all articles that will be published in the journal. After a manuscript is technically edited and formatted, it will be removed from the “Just Accepted” Web site and published as an ASAP article. Note that technical editing may introduce minor changes to the manuscript text and/or graphics which could affect content, and all legal disclaimers and ethical guidelines that apply to the journal pertain. ACS cannot be held responsible for errors or consequences arising from the use of information contained in these “Just Accepted” manuscripts.

ACS Photonics is published by the American Chemical Society. 1155 Sixteenth Street N.W., Washington, DC 20036 Published by American Chemical Society. Copyright © American Chemical Society. However, no copyright claim is made to original U.S. Government works, or works produced by employees of any Commonwealth realm Crown government in the course of their duties.

Page 1 of 25

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

ACS Photonics

Background Estimation and Correction for High-Precision Localization Microscopy Ching-Ya Cheng1, Chia-Lung Hsieh1,* 1 Institute of Atomic and Molecular Sciences (IAMS), Academia Sinica, Taipei 10617, Taiwan * [email protected]

ABSTRACT Localization of single nano-sized light emitter has substantial applications in bioimaging. The accuracy and precision of localization are limited by the noise and the heterogeneous background superimposed on the signal. While the effects of noise are well recognized, influence of background is less addressed. Proper background correction not only provides more accurate localization data but also enhances the sensitivity of detection. Here, we demonstrate a new approach to background correction by estimating and removing the heterogeneous but stationary background from a series of images containing a spatially moving signal. Our approach exploits the correlated signal information encoded in the neighboring pixels governed by the point-spread function of the measurement system. This new approach makes it possible to obtain the background even when the total displacement of the signal is sub-diffraction limit throughout the observation, the scenario where previous methods become invalid. We characterize our approach systematically with different types of signal motions at various signal-to-noise ratios in numerical simulations. We then verify our method experimentally by recovering the nanoscopic displacements of single gold nanoparticle moving in a specified pattern and single virus particle randomly diffusing on a cell surface. The source code of our algorithm written in MATLAB is provided together with a sample data set. Our approach has immediate applications in high-precision optical localization measurements. KEYWORDS: localization microscopy, single-particle tracking, background correction, background estimation, interferometric scattering microscopy, coherent brightfield microscopy

1

ACS Paragon Plus Environment

ACS Photonics

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

The concept of localizing single nano-sized light emitter under an optical microscope has been the foundation of single-molecule tracking and superresolution imaging1-3. It has immense applications to biological studies4, including the recent advances in intracellular machinery5,6, membrane dynamics7-9, viral infection10-12, sensing13-18, and drug delivery19. In localization microscopy, the position of an isolated nano-emitter is estimated from its optical image, defined by the point-spread function (PSF) of the microscope. The size of the PSF of a typical optical microscopy is a diffraction-limited airy disk whose size is about half of the optical wavelength. Importantly, the position of the disk center, corresponding to the location of the emitter, can be estimated with a precision much better than the size of the spot. In practice, the localization precision is determined by the signal-to-noise ratio (SNR): higher precision can be realized when a greater SNR is provided. In the ideal scenario where the photon noise is the dominant source of noise, the SNR is determined by the number of detected signal photons20. Unfortunately, in the experiments, the number of signal photons is often limited, resulting in limited SNR. Fluorescence imaging particularly suffers from the photophysics and photochemistry of the fluorophores (photobleaching and saturation) which put a stringent limit on the amount of available signal. Motivated by revealing structures and fast dynamics at the nanoscale, scientists have tried to improve the localization precision and the temporal resolution by using linear scattering as the signal. The linear scattering signal is indefinite which offers the opportunity to observe rapid dynamics with a high precision that are beyond the reach of fluorescence. Among the scattering-based approaches, imaging-based interferometry with laser illumination, e.g., interferometric scattering (iSCAT) microscopy21-23 and coherent brightfield (COBRI) microscopy11,24, have shown great promise of probing nano-sized objects with high sensitivity and at high spatiotemporal resolution. Using these imaging modalities, small gold nanoparticles22,23,25,26, virus particles10-12, intracellular cell vesicles24, membrane nanodomains27, and unlabeled protein molecules13-16 can be directly visualized via intrinsic scattering. Interferometric detection offers many advantages, including shot-noise limited sensitivity26 and three-dimensional localization11,24,28. Unlike fluorescence imaging which is basically background-free, the success of high-precision scattering-based imaging relies critically on the background correction. When the sensitivity increases, ambient weak background scattering becomes detectable which complicates the detection of signal of interest. Even in the experiments where the background scattering of the samples was reduced as much as possible, inevitable stray light from the optical components and imperfect illuminations readily prevent us from detecting the weakest signal in the raw image. Therefore, a general method of background estimation and correction has become imperative. In the previous studies, scattering background was measured when the signal was not present9-15,24-27. In the simplest case, a background image can be recorded before the signal appears in the region of interest, e.g., before the particle or the molecule attaches to the substrate under 2


Page 2 of 25

Page 3 of 25

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

ACS Photonics

observation13-16. In the cases where the object of interest remains in the field of view throughout the recording9,11,24-26, it may be possible to “extract” a background image from the recorded video by examining the data pixel by pixel. In principle, one should be able to obtain the background value of every pixel as long as the object moves far enough so that each pixel is not occupied by the object for at least one frame. In practice, it requires sufficient number of observations of the background value of the pixel throughout the recording in order to decide which intensity value corresponds to the background. When the above condition is met, the background value can be reasonably estimated by finding the median value of the recorded intensities of individual pixels25. This method works well for finding the static background that usually comes from the undesired reflections of stationary optical interfaces. Interestingly, we recently extended this method to calculate and then remove scattering background created by live cells11. By removing the relatively stationary cellular background, single diffusive virus particles can be clearly observed on the cell plasma membranes and their dynamics were investigated with unprecedented clarity11. Despite the recent success of background estimation and correction in scattering-based imaging, it is still challenging to obtain the background in many experiments. In fact, the previous approaches of background estimation would fail if some pixels were occupied by the signal throughout the observation time. In localization microscopy, that corresponds to the situation where the particle of interest only moves over a short distance, smaller than the size of the diffraction-limited PSF. As a result, the recorded intensities of some pixels are always affected by the signal of the particle, leaving no chance for a glance of the background. We notice that this is often the situation when investigating nanoscopic structures and dynamics: the particle moves locally over a small area of interest that is only hundreds of nanometers throughout the observation, possibly representing a confinement zone on the membrane29, or the movement of a rotary motor protein30. Longer observation time is not of much help because although the particle may migrate over a larger distance in a prolonged observation time, the background usually changes significantly during the long observation. Without proper background estimation, larger labeling particles were needed in order to have much stronger signal than the background. Unfortunately, labeling of larger particles introduces greater loading, experiences stronger steric hindrance, and is prone to labeling artifacts31. We note that there have been many strategies of background estimation and subtraction in the field of computer vision32,33. The considerations and tasks of computer vision are generally very different from localization microscopy (Supplementary Note 1). To the best of our knowledge, none of the existing method is readily applicable to resolve the localized motion described above. In this work, we report a new approach to estimating the stationary background for correction in high-precision localization microscopy. The strategy is to exploit the knowledge of PSF of the optical system. Through an iterative algorithm, signal that is described by a moving PSF is separated from the static background. Our method is able to find the correct background even when the 3


ACS Photonics

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

displacement of the particle is smaller than the PSF. We characterize our approach with simulated data of different types of motions (regular stepwise motion and Brownian diffusive motion) at various SNRs. We also verify our method with experimental data of gold nanoparticle moving in the specified patterns and virus particle diffusing on the cell membrane. PRINCIPLE OF BACKGROUND ESTIMATION Our concept of distinguishing the signal from the background is based on their different fluctuating signatures. Consider a situation where the signal moves over space and the background is relatively stationary, a common scenario of localization microscopy and single-particle tracking. It is often possible to perceive the presence of the signal in the raw image, but the SNR is too low for high-precision localization. Movie S1 shows an example of a moving signal spot in the presence of a heterogeneous background (a snapshot is displayed in Fig. 1a). Here the background is a typical scattering image of a biological cell under COBRI microscopy11 (Fig. 1b). The signal in Movie S1 is a simulated PSF which moves on 11 x 11 grids with a grid size of 20 nm. Poisson photon noise is added to the simulated image (see the details of generating simulated images in Methods). The SNR of individual image, defined as the ratio between the amplitude of the PSF and the standard deviation of the pixel intensities of the background, is as low as 1.3. Direct detection and localization of the signal in the raw image is very difficult and susceptible to severe bias due to the heterogeneous background, leading to a wrong estimation of the true position of the signal. If the background image is available, one could drastically enhance the SNR by removing the background from the raw image (illustrated in Fig. 1c where the background in Fig. 1b is removed from the raw image Fig. 1a). However, in the case shown in Movie S1, the overall movement of the signal in the video is smaller than the size of the PSF, and therefore some pixels are occupied by the signal throughout the video, leaving no direct access to their background values. Our task is to find the background of all pixels from the video and then remove the background in order to achieve high-precision localization of the signal. We propose a new strategy of extracting the background from the video. Instead of examining the data pixel by pixel independently as in the previous studies, our new method exploits the information encoded in the neighboring pixels. Specifically, we optimize the estimation of background by minimizing the residual error of fitting the background-corrected image with a specified PSF through an iterative process (see Fig. 2 for the flowchart and Methods for the detailed procedure of background optimization). This optimization process repeats itself until the estimation converges (see Methods for the criteria of convergence). Intuitively, since the signal spot moves over space, every pixel is occupied differently by the signal PSF throughout the video. Even when some pixels are affected by the signal PSF all the time (and thus has no chance to peer their background values directly), their neighboring pixels are likely to be less influenced by the signal because the moving nature of the signal. For these neighboring pixels, better estimation of the 4


Page 4 of 25

Page 5 of 25

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

ACS Photonics

background values can be made. These good estimations of the neighboring pixels eventually help to determine the background value of the pixels that are originally inaccessible, based on the fact that their intensities are correlated through the signal PSF. The MATLAB source code of our algorithm and a sample data set are available as Supporting Information. A technical note of the algorithm can be found in Supplementary Note 2. RESULTS AND DISCUSSION Background extraction and correction for simulated stepwise and Brownian motions. We first assume that the PSF is known explicitly (this is a helpful but non-essential simplification that we will relax later). In most localization measurements, the PSF is approximated as a 2D Gaussian function34. This approximation is generally applicable (although a better model would be an airy disk) because the SNR is usually not sufficient to see the side lobes that originate from the sharp cutoff in the collected propagating spatial frequencies by a circular aperture. It should be noted that, for high-precision localization, the PSF could deviate from a 2D Gaussian function, depending on the polarization and orientation of the nano-emitter35. In the case of interferometric imaging where the interference of the signal electric field with a reference field is the dominant signal, the side lobes become more apparent and the PSF consists of concentric rings11. We model the PSF (centered at the origin) in interferometric imaging as the square root of an airy disk: PSF =

(1)

where is the distance to the center of the PSF, ∙ is the Bessel function of order one, and are variables that describe the amplitude and the width of the PSF, respectively. When the PSF is known, both values of and are given. This is usually a valid assumption when the motion of the particle is in the 2D focal plane. When the particle moves in the vertical direction, the amplitude of the signal () changes due to interference, followed by a slight change in spot size ()11. We examine our algorithm of background extraction with the series of simulated images shown in Movie S1. In the video, a PSF with a contrast of -0.05 is superimposed on a cellular background image followed by noise addition (see Methods for generation of simulated images). The contrast of -0.05 in COBRI imaging roughly corresponds to the signal of a 40 nm gold nanoparticle. The particle moves on 11 x 11 grids with a grid size of 20 nm (see schematics in Fig. 3a). At each grid intersection, the particle stays for 100 steps. In this case, direct localization of the particle in the raw image is not possible because of the low SNR. We use our algorithm for background extraction. It finds a background image after 13 iterations. By correcting the raw images with this estimated background (see Movie S2), we obtain the reconstructed trajectory of the particle that appears to be correct (Fig. 3b, blue). To evaluate the accuracy of the trajectory, we also obtain the true trajectory of the particle by using the original background image for correction (Fig. 3b, yellow). The difference between these two trajectories is very small (0.96 nm in X and 0.86 nm in Y, rms), indicating that the 5


ACS Photonics

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

background estimation is very accurate and successful. We confirm that our estimated background image is almost identical to the true background image (normalized standard deviation = 0.021%). To carefully examine the accuracy of the recovered trajectory, Fig. 3c displays the error image, defined as the difference between the two trajectories in Fig. 3b. It appears that the error increases slightly as the signal approaches to the border of the trajectory, presumably due to the decreasing background sampling at the edge of the exploration area. We repeat the same analysis but relax the constraint that the exact shape of PSF is known by leaving the amplitude and width of the PSF (i.e., and in equation (1)) as two independent fitting parameters with loose constraints (within a factor of two). Again, the background image is properly estimated, giving a correct reconstructed trajectory (Fig. 3d, red) with an error of 0.33 nm in X and 0.34 nm in Y (rms). Fig. 3e plots the error image of the two trajectories in Fig. 3d, showing a slightly smaller error than that of known PSF in Fig. 3c. We note that the accuracy of background correction depends on the exact background under analysis. More characterization is discussed in the next section. In general, our algorithm is able to estimate the background properly as long as the PSF could be modeled by the fitted function, and it does not require a fixed PSF shape. We refer to this scenario as “unknown PSF”. In the rest of the work, all the backgrounds are estimated with the assumption of unknown PSF. We verify the convergence of our algorithm by examining the localization error at each iteration (Fig. 4). The previous two cases are considered: known PSF and unknown PSF. In the beginning, the backgrounds are poorly estimated in both cases and therefore the errors are large (> 30 nm). The errors decrease rapidly in the case of known PSF (by a factor of 25 after only 7 iterations). In the case of unknown PSF, same reduction in error is reached after ~80 iterations. The correction slowly converges to the optimal. We terminate the iteration when the algorithm does not provide a better estimated background, judged by the total residual error of the fitting (see Methods). We note that using a 2D Gaussian function to fit the data is problematic especially at high SNR because the 2D Gaussian function cannot describe the side lobe rings of the PSF that would be misinterpreted as the background (see the effect quantitatively in Fig. S1). We also examine the background estimation method of temporal median filtering, a common strategy used in many previous studies. When the signal stays locally throughout the total observation time, the local pixels are always affected by the signal and thus their background values are not directly accessible. The temporal median filtering gives wrong estimations of those pixels by misinterpreting part of the signal into the background. The background values of those pixels are very critical for localization because they are at the position where the PSF explores. As the result, the reconstructed trajectory of temporal median filtering is distorted severely (Fig. S2).

6


Page 6 of 25

Page 7 of 25

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

ACS Photonics

Characterization of the background estimation algorithm We systematically characterize our algorithm under various conditions. We find that the performance our method is most sensitive to the SNR and the maximal displacements. Ten cell images captured by COBRI microscopy are used as background images (see Fig. S3). In addition, for each background, we evaluate the performance with the signal contrast of -0.05, -0.07, -0.1, -0.15, -0.2, -0.3, and -0.4. Each combination of the background image and the signal contrast gives a SNR. The SNR is defined as the ratio of the signal contrast to the standard deviation of the background image. In our simulations, the SNR ranges from ~1 to 10. The signal moves on 11 x 11 grids with a grid size of 20 nm as shown in Fig. 3a. In all cases, our algorithm succeeds in finding the backgrounds. Reconstructed trajectories are then obtained by localizing the signal spots in the background-corrected videos. The difference between the reconstructed trajectory and the noise-free pre-defined motion as a function of SNR is displayed in Fig. 5a. It shows that the reconstruction becomes more accurate at higher SNR. This is mostly due to the reduced effect of photon noise in localization when the SNR increases. We note that a more relevant quantity for evaluating the performance of background estimation is the difference between the reconstructed trajectories by using the estimated background the true background. Their difference is independent to the photon shot noise (as the exact photon shot noise is considered in both cases of the same data set). The difference of the localizations between the estimated background and true background is shown in Fig. 5b: 97% of the cases (68/70) show the localization errors less than 1 nm (rms), and 3% of the cases show localization errors of 1–2 nm (rms). Compared to Fig. 5a, the result of Fig. 5b is less SNR-dependent, indicating that our algorithm is able to estimate background of equally good quality at various SNR (as low as SNR of 1). Interestingly, we notice that, in some case, our method works well even when the SNR is slightly lower than 1, which is counterintuitive. The reason for this success is that our algorithm considers information in many images (instead of a single one) during background estimation and correction. Once the background is properly estimated, the SNR of the background-corrected image is enhanced significantly, facilitating the accurate and precise localization. We find that the number of iteration required for finding the background increases when the SNR decreases (Fig. 5c). This is because the initial background estimation is more wrong at a lower SNR and it takes longer to converge to the correct one. We further examine our algorithm by decreasing the grid size. Fig. S4 shows the localization errors with grid sizes of 15, 10, 5 nm at different SNRs. As the movement of the signal becomes smaller, it is increasingly difficult to distinguish the signal from the stationary background. When compared to the results of the true background, the portion of accurate localizations (error < 1 nm) drops as the grid size decreases (the portions are 97%, 83%, 43%, and 16% for the grid sizes of 20, 15, 10, and 5 7


ACS Photonics

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

nm, respectively). Besides the effect of grid size, larger localization error happens at lower SNR. We also note that the number of iteration required for convergence increases at a decreasing grid size (Fig. S4). The portions of optimizations that converge within 500 iterations are 100%, 91%, 57%, and 56% for the grid sizes of 20, 15, 10, and 5 nm, respectively. About 10% of the optimizations do not converge after 2,000 iterations at the grid size of 10 nm and 5 nm. We also check the effect of the spatial sampling density, the size of the image (region of interest, ROI), and the total number of observations (see the results in Fig. S5). They play little role in our algorithm compared to the SNR and the maximal displacement. Precise reconstruction of experimental stepwise and Brownian motions via background estimation and correction. We experimentally verify our background estimation and correction. We imaged single 40 nm gold nanoparticle deposited on coverslip, immersed in water, under COBRI microscopy. The experimental setup of our COBRI microscopy is described in the Methods. The lateral position of the sample was controlled by a piezo-stage with sub-nanometer resolution. The raw image of 40 nm gold nanoparticle is shown in Fig. 6a. Although the sample was clean and simple, the particle is barely seen in the raw image due to the presence of a heterogeneous background. This illustrates the typical scenario in scattering-based interferometric imaging where inevitable reflections of optical components contribute to the background and they further interfere with each other, leading to interference fringes of various patterns36. We then moved the sample laterally in a stepwise manner on 11 x 11 grids with a grid size of 20 nm (as shown in Fig. 3a) during which a video was recorded (Movie S3, containing 15,060 frames). We applied our algorithm to estimate the background from the recorded video. An estimated background was found after 113 iterations. By correcting this estimated background from the raw image, the particle was clearly observed (Fig. 6a), showing a contrast of ~-0.05. The SNR in the raw image is estimated to be 1.8, and our background correction enhances the SNR to a value of 7. The background-corrected video is shown in Movie S4. For verification, we tried to measure a background by imaging an empty area on the sample without the particle through moving the sample laterally by a few microns26. This method measures the static background including those created by the illumination inhomogeneities, spurious back-reflections, and etalon fringe patterns caused by the cover window of the camera. If the sample itself contributes to the background (e.g., dirty supporting coverslip), moving the sample laterally would give a different background that does not correspond to the original measurement. In our case of COBRI imaging, we make sure the sample has minimal contribution to the background, so the measured background should be close to the true one. We found that the experimentally 8


Page 8 of 25

Page 9 of 25

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

ACS Photonics

measured background is very similar to the estimated one (normalized standard deviation is 0.27%). We compare the reconstructed trajectories obtained by localizing the particle in the raw video and in the two corrected videos where the measured background and the estimated background are used respectively (Fig. 6b-d). The raw video gives a biased, distorted trajectory (Fig. 6b, light blue), while the both corrected videos give orthogonal, accurate, stepwise movements (Fig. 6c, yellow and red). Quantitatively, when using the trajectory corrected with the measured background as the reference, the rms localization error of no correction is 12.7 nm in X and 16.9 nm in Y. The rms localization error is reduced to 6.6 nm in X and 4.5 nm in Y when corrected with the estimated background. To examine carefully the difference between the reconstructed trajectories in Fig. 6b and 6c, we plot their error images in Fig. 6d. While the error is evidently reduced after background correction, the residual error is larger compared to the simulated results of Fig. 3e. This larger error of the experimental data may come from the imperfect signal PSF. We note that, the PSF in the experiment is not always perfectly symmetric, possibly due to the focusing condition, the shape of the particle, the polarization effect, and the non-perfect optical system (astigmatism, aberration, or misalignment). The discrepancy between the experimental PSF and the model PSF leads to an error in the background estimation. Another possible explanation is that the measured background may not serve as the exact true background, as the background may change slightly when moving the sample laterally. Nevertheless, it is clear that our background estimation and correction greatly improves the accuracy of localization. Next, we test our algorithm by laterally moving the particle in a pre-programed random walk, recorded in 3,830 frames. The maximal distance explored by the particle was ~200 nm. Our algorithm found a background after 456 iterations. Similarly, we experimentally acquired a background image for verification by imaging an empty area without the particle. The estimated background and measured background are very close (normalized standard deviation is 0.31%). The comparison of reconstructed trajectories obtained from the raw video and the corrected videos are shown in Fig. 6e-g. Again, the results of measured background and estimated background were very similar, while the result without correction shows distortion. The localization error of no correction is 13.7 nm in X and 6.4 nm in Y, and it becomes 4.9 nm in X and 6.4 nm in Y after correction with the estimated background. We thus conclude that our approach works well in reconstruction of diffusive motion of nanoparticles with experimental data. Verification of background extraction and correction for tracking single virus particle in live cells. We then use our algorithm to reconstruct the motion of single virus particle on the surface of live cells. We previously used COBRI microscopy to capture the events of single vaccinia virus particles landing on the plasma membrane of live cells in a continuous manner11. When the virus attached to the membrane, it became locally confined in a zone of hundreds of nanometers. Within the confinement, the virus particle diffused rapidly. The cell structures appeared as a heterogeneous 9


ACS Photonics

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

background in the image. When taking images at a sufficient acquisition rate (5,000 frames per second as demonstrated), the cell background was relatively stationary compared to the spatially moving viral signal. The optical contrast of the virus particle was ~-0.2. In the previous studies, we were limited to use the temporal median filtering or the cell image taken before the virus landing as the background. Once the virus particle became strongly confined for an extended period of time, the previous approaches to background correction failed. We apply the algorithm of background estimation described in this work to the viral images measured in live cells (raw video shown in Movie S5). For the purpose of verification, we intentionally choose the data where the cell background was available by taking a background image right before the virus attachment (not shown in the video). Fig. 7a displayed the raw image, the estimated cell background, and the background-corrected virus image. Our background estimation algorithm finds a cell background in 25 iterations. Movie S6 shows the corrected video by using the estimated background where the SNR is enhanced roughly by an order of magnitude. The normalized standard deviation between the measured and the estimated background and is small, ~1.35%. We compared the reconstructed trajectories obtained by directly localizing the virus particle in the raw images (no correction) and in the background-corrected images (Fig. 7b-d). Without surprises, the two corrected trajectories are much similar (rms difference of 6.7 nm in X and 2.9 nm in Y), while the non-corrected trajectory is biased and distorted (rms error of 10.6 nm in X and 21.5 nm in Y). CONCLUSION We demonstrated a method to estimate and correct the background for high-precision localization of single nano-sized particle via interferometric scattering-based optical imaging. Compared to the existing approaches, our method takes advantage of the knowledge of the PSF of the measurement system, and therefore exploits the correlated information in neighboring pixels. Using our methods, we succeeded in background corrections and high-precision localizations in the scenarios where previously unavailable. We characterized our approach systematically in simulation and examined our approach with experimental data. Signal was successfully detected at SNR as low as 1 in raw images and its nanoscopic motion was recovered accurately after background correction. We note that our approach to background correction is general and applicable to measurements of dynamic diffraction-limited signals in the presence of a relatively stationary background. While demonstrated in 2D in this work, the concept readily works in 1D spectroscopy measurements and 3D localization. Future application is beyond optical measurements (e.g., acoustics and electron measurements). The essential criteria of our method are: (1) knowledge of the object (defined by a model function with fitting parameters), and (2) sufficient observations (>100 frames) in which background is static and signal is spatially moving (or vice versa). When the above criteria are met, our method is able to resolve highly localized motion (maximal displacement as small as ~50 nm, corresponding to ~1/5 of 10


Page 10 of 25

Page 11 of 25

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

ACS Photonics

the PSF). We expect that a more general approach that relaxes the above-mentioned constraints may exist. Our current approach has immediate applications in improving sensitivity and accuracy in interferometric scattering-based optical microscope imaging, especially in bioimaging.

11


ACS Photonics

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 12 of 25

METHODS Generation of simulated images In interferometric imaging, the image records the interference between signal and reference. The detected intensity ( ) is contributed by three terms: the signal intensity ( ), the reference intensity (! ), and their interference (2# ! cos '): = + ! + 2# ! cos '

(2)

where ' is the phase difference between the signal and the reference fields. The phase ' is generally chosen to be 0 or 180 degrees in order to maximize the magnitude of the interference term. In this work, we chose ' = 180°. When imaging small particles, the magnitude of signal intensity is small, much smaller than that of the interference term and thus negligible. Therefore, the detected intensity can be approximated as: ≈ ! − 2# ! .

(3)

In our simulation, we used images of HeLa cells taken by COBRI microscopy as our background (i.e., ! ). We then generated a signal (i.e., ) with an amplitude that corresponds to a desirable contrast (defined as −2# ! /! = −2# ⁄! ). The width of the PSF (i.e., the variable in Eq. (1)) was chosen to match with our experimental data. A simulated image was then created by Eq. (3), with a pixel size of 48 nm. We then added Poisson noise to every pixel based on the detected number of photons, estimated by the full width capacity of the camera. We generated a series of images where the signal moved over the space in a specified pattern. Procedures of iterative background estimation We first cropped an image of 20 x 20 pixels in which the particle of interest was roughly centered. This cropped image was large enough to contain the main side lobe of the PSF. We normalized this image by the median of all pixel values. The initial guess of the background image was a uniform image of all ones. We then attempted to detect particles from the raw image by smoothing and thresholding.37 We did not put any constrain on the number of detected signal spots in the image. False detection was inevitable at low SNR. For each detected particle, we fitted it with the square root of an airy disk (Eq. (1)). From each fitting, a residual error image (the ratio between the raw image and the fitted image) was calculated and considered as a potential correction to the background. We define the residual error image as the “ratio” not the “difference” between the raw image and the fitted image because the strength of the signal is easier to measure in the normalized image of interferometric imaging. Suppose the video contained N detected particles in total (in M frames), we then had N potential corrections. If the detected particle was false (belonged to the stationary background), the estimated particle position would be almost fixed throughout the video (fluctuating only due to the photon noise). We discarded the corrections from those particles whose displacements were too small (the standard deviation of the localizations was smaller than the localization precision defined by the accuracy of the nonlinear least-squares fitting38). As the real signal spot moved over space, the corrections proposed by the signal from each frame would not be 12


Page 13 of 25

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

ACS Photonics

identical. For each pixel, there were now many proposed correction values (typically a little bit more than the number of frames of the video). Among these proposed values, we chose their median as our correction to the estimated background value. To this point, we obtained the first estimated background. The algorithm then used this estimated background to correct the raw images and repeated the fitting and optimization. Criteria of convergence During the iterative optimization, we monitored the evolution of two quantities to determine whether the optimal background was obtained. The first quantity was the difference in reconstructed trajectories of the two consecutive iterations. If the difference (absolute distance) between all localizations of the two trajectories (containing ~12,000 steps) was sufficiently small (< 10-3 nm for each localization on average), we consider the optimization converged and thus terminated the iteration. The other quantity was the quality of fitting. If the total residual error of the fitting (of all images) was not reduced further, we considered the current estimated background was optimal and thus terminated the iteration. We terminate the optimization when one of the conditions mentioned above was met, or when the number of iteration reached 2,000. Experimental setup and hardware The experimental data of gold particle and virus particle were acquired by COBRI microscopy developed in our lab. The setup was described in details previously11. Briefly, our COBRI microscope was an inverted microscope, using a laser at 532 nm wavelength as the light source. The sample was illuminated from the top by focusing the laser on the sample with a lens. The sample was placed on a 3-axis piezo system (P-545, Physik Instrumente) for precise motion control. The transmitted signal was collected by an oil-immersion objective (UPLSAPO 100 OX, NA1.4, Olympus), and projected to a high-speed CMOS camera (Phantom v711, Vision Research). The COBRI video was recorded at 1,000 fps for the gold particle and 5,000 fps for the virus particle. The video was stored in the hard disk for further analysis. We used a PC (Dell Optiplex 9010, Intel Core i7-3770 CPU at 3.4 GHz, 8GB RAM) for background calculation. For an image sequence of 1,210 frames of 20 x 20 pixels, it takes ~1,000 seconds for our algorithm to estimate a background image after 141 iterations. The computation time increases as the number of iterations and the resolution of image increase. Compared to other methods, e.g., temporal median filtering, our method requires more computation power because of the iterative PSF fitting and the iterative background optimization processes. ASSOCIATED CONTENT Supporting Information The Supporting Information is available free of charge on the ACS Publications website at DOI: 10.1021/acsphotonics.xxxxxxx. Discussion of the methods of background estimation and correction developed for computer vision 13


ACS Photonics

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

and their applications in localization microscopy; technical note of the background estimation algorithm; effect of using a 2D Gaussian function to fit the PSF of a square root of an Airy disk in interferometric microscopy; evaluation of background estimation by temporal median filtering; ten cell background images used in the simulations; performance of the background-estimation algorithm of smaller gird size; effects of spatial sampling density, the size of ROI, and the number of observations in our algorithm (PDF) Simulated raw video of a particle moving on 11 x 11 grids (AVI) Background-corrected simulated video of a particle moving on 11x11 grids (AVI) Experimentally acquired raw video of a gold nanoparticle moving on 11x11 grids (AVI) Background-corrected experimentally acquired video of a gold particle moving on 11x11 grids (AVI) Raw video of a vaccinia virus particle diffusing on a cell membrane (AVI) Background-corrected video of a vaccinia virus particle diffusing on a cell membrane (AVI) MATLAB code and sample data set (ZIP) ACKNOWLEDGEMENTS This work was supported by the Nano Program of Academia Sinica and grant from the Ministry of Science and Technology, Taiwan (105-2112-M-001-016-MY3).

14


Page 14 of 25

Page 15 of 25

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

ACS Photonics

REFERENCES (1) Betzig, E.; Patterson, G. H.; Sougrat, R.; Lindwasser, O. W.; Olenych, S.; Bonifacino, J. S.; Davidson, M. W.; Lippincott-Schwartz, J.; Hess, H. F., Imaging Intracellular Fluorescent Proteins at Nanometer Resolution. Science 2006, 313, 1642-1645. (2) Manley, S.; Gillette, J. M.; Patterson, G. H.; Shroff, H.; Hess, H. F.; Betzig, E.; Lippincott-Schwartz, J., High-Density Mapping of Single-Molecule Trajectories with Photoactivated Localization Microscopy. Nat Meth 2008, 5, 155-157. (3) Patterson, G. H.; Davidson, M. W.; Manley, S.; Lippincott-Schwartz, J., Superresolution Imaging Using Single-Molecule Localization. Annu. Rev. Phys. Chem. 2010, 61, 345-367. (4) Huang, B.; Bates, M.; Zhuang, X. W., Super-Resolution Fluorescence Microscopy. Annu. Rev. Biochem. 2009, 78, 993-1016. (5) Sun, W.; Gu, Y.; Wang, G.; Fang, N., Dual-Modality Single Particle Orientation and Rotational Tracking of Intracellular Transport of Nanocargos. Anal. Chem. 2012, 84, 1134-1138. (6) Mehta, S. B.; McQuilken, M.; La Riviere, P. J.; Occhipinti, P.; Verma, A.; Oldenbourg, R.; Gladfelter, A. S.; Tani, T., Dissection of Molecular Assembly Dynamics by Tracking Orientation and Position of Single Molecules in Live Cells. Proceedings of the National Academy of Sciences 2016, 113, E6352-E6361. (7) Cutler, P. J.; Malik, M. D.; Liu, S.; Byars, J. M.; Lidke, D. S.; Lidke, K. A., Multi-Color Quantum Dot Tracking Using a High-Speed Hyperspectral Line-Scanning Microscope. PLoS One 2013, 8, e64320. (8) Kusumi, A.; Tsunoyama, T. A.; Hirosawa, K. M.; Kasai, R. S.; Fujiwara, T. K., Tracking Single Molecules at Work in Living Cells. Nat. Chem. Biol. 2014, 10, 524-532. (9) Wu, H.-M.; Lin, Y.-H.; Yen, T.-C.; Hsieh, C.-L., Nanoscopic Substructures of Raft-Mimetic Liquid-Ordered Membrane Domains Revealed by High-Speed Single-Particle Tracking. Sci. Rep. 2016, 6, 20542. (10) Kukura, P.; Ewers, H.; Müller, C.; Renn, A.; Helenius, A.; Sandoghdar, V., High-Speed Nanoscopic Tracking of the Position and Orientation of a Single Virus. Nat. Methods 2009, 6, 923-927. (11) Huang, Y.-F.; Zhuo, G.-Y.; Chou, C.-Y.; Lin, C.-H.; Chang, W.; Hsieh, C.-L., Coherent Brightfield Microscopy Provides the Spatiotemporal Resolution to Study Early Stage Viral Infection in Live Cells. ACS Nano 2017, 11, 2575-2585. (12) Goldfain, A. M.; Garmann, R. F.; Jin, Y.; Lahini, Y.; Manoharan, V. N., Dynamic Measurements of the Position, Orientation, and DNA Content of Individual Unlabeled Bacteriophages. The Journal of Physical Chemistry B 2016, 120, 6130-6138. (13) Ortega Arroyo, J.; Andrecka, J.; Spillane, K. M.; Billington, N.; Takagi, Y.; Sellers, J. R.; Kukura, P., Label-Free, All-Optical Detection, Imaging, and Tracking of a Single Protein. Nano Lett. 2014, 14, 2065-2070. (14) Piliarik, M.; Sandoghdar, V., Direct Optical Sensing of Single Unlabelled Proteins and 15


ACS Photonics

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Super-Resolution Imaging of Their Binding Sites. Nat. Commun. 2014, 5, 4495. (15) Liebel, M.; Hugall, J. T.; van Hulst, N. F., Ultrasensitive Label-Free Nanosensing and High-Speed Tracking of Single Proteins. Nano Lett. 2017, 17, 1277-1281. (16) Cole, D.; Young, G.; Weigel, A.; Sebesta, A.; Kukura, P., Label-Free Single-Molecule Imaging with Numerical-Aperture-Shaped Interferometric Scattering Microscopy. ACS Photonics 2017. (17) Avci, O.; Campana, M. I.; Yurdakul, C.; Selim Ünlü, M., Pupil Function Engineering for Enhanced Nanoparticle Visibility in Wide-Field Interferometric Microscopy. Optica 2017, 4, 247-254. (18) Faez, S.; Lahini, Y.; Weidlich, S.; Garmann, R. F.; Wondraczek, K.; Zeisberger, M.; Schmidt, M. A.; Orrit, M.; Manoharan, V. N., Fast, Label-Free Tracking of Single Viruses and Weakly Scattering Nanoparticles in a Nanofluidic Optical Fiber. ACS Nano 2015, 9, 12349-12357. (19) Schuster, B. S.; Ensign, L. M.; Allan, D. B.; Suk, J. S.; Hanes, J., Particle Tracking in Drug and Gene Delivery Research: State-of-the-Art Applications and Methods. Adv. Drug Delivery Rev. 2015, 91, 70-91. (20) Thompson, R. E.; Larson, D. R.; Webb, W. W., Precise Nanometer Localization Analysis for Individual Fluorescent Probes. Biophys. J. 2002, 82, 2775-2783. (21) Lindfors, K.; Kalkbrenner, T.; Stoller, P.; Sandoghdar, V., Detection and Spectroscopy of Gold Nanoparticles Using Supercontinuum White Light Confocal Microscopy. Phys. Rev. Lett. 2004, 93, 037401. (22) Jacobsen, V.; Stoller, P.; Brunner, C.; Vogel, V.; Sandoghdar, V., Interferometric Optical Detection and Tracking of Very Small Gold Nanoparticles at a Water-Glass Interface. Opt. Express 2006, 14, 405-414. (23) Ortega-Arroyo, J.; Kukura, P., Interferometric Scattering Microscopy (iSCAT): New Frontiers in Ultrafast and Ultrasensitive Optical Microscopy. Phys. Chem. Chem. Phys. 2012, 14, 15625-15636. (24) Huang, Y.-F.; Zhuo, G.-Y.; Chou, C.-Y.; Lin, C.-H.; Hsieh, C.-L., Label-Free, Ultrahigh-Speed, 3D Observation of Bidirectional and Correlated Intracellular Cargo Transport by Coherent Brightfield Microscopy. Nanoscale 2017. (25) Hsieh, C. L.; Spindler, S.; Ehrig, J.; Sandoghdar, V., Tracking Single Particles on Supported Lipid Membranes: Multimobility Diffusion and Nanoscopic Confinement. J. Phys. Chem. B 2014, 118, 1545-1554. (26) Lin, Y.-H.; Chang, W.-L.; Hsieh, C.-L., Shot-Noise Limited Localization of Single 20 nm Gold Particles with Nanometer Spatial Precision within Microseconds. Opt. Express 2014, 22, 9159-9170. (27) de Wit, G.; Danial, J. S. H.; Kukura, P.; Wallace, M. I., Dynamic Label-Free Imaging of Lipid Nanodomains. Proc. Natl. Acad. Sci. U. S. A. 2015, 112, 12299-12303. (28) Krishnan, M.; Mojarad, N.; Kukura, P.; Sandoghdar, V., Geometry-Induced Electrostatic 16


Page 16 of 25

Page 17 of 25

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

ACS Photonics

Trapping of Nanometric Objects in a Fluid. Nature 2010, 467, 692-695. (29) Kusumi, A.; Suzuki, K. G. N.; Kasai, R. S.; Ritchie, K.; Fujiwara, T. K., Hierarchical Mesoscale Domain Organization of the Plasma Membrane. Trends Biochem. Sci. 2011, 36, 604-615. (30) Ueno, H.; Nishikawa, S.; Iino, R.; Tabata, K. V.; Sakakihara, S.; Yanagida, T.; Noji, H., Simple Dark-Field Microscopy with Nanometer Spatial Precision and Microsecond Temporal Resolution. Biophys. J. 2010, 98, 2014-2023. (31) Clausen, M. P.; Lagerholm, B. C., The Probe Rules in Single Particle Tracking. Curr. Protein Pept. Sci. 2011, 12, 699-713. (32) Panahi, S.; Sheikhi, S.; Hadadan, S.; Gheissari, N., Evaluation of Background Subtraction Methods, Digital Image Computing: Techniques and Applications 2008, 357-364. (33) Piccardi, M., Background Subtraction Techniques: a Review. IEEE International Conference on Systems, Man and Cybernetics 2004, 4, 3099-3104. (34) Chenouard, N.; Smal, I.; de Chaumont, F.; Maska, M.; Sbalzarini, I. F.; Gong, Y.; Cardinale, J.; Carthel, C.; Coraluppi, S.; Winter, M.; Cohen, A. R.; Godinez, W. J.; Rohr, K.; Kalaidzidis, Y.; Liang, L.; Duncan, J.; Shen, H.; Xu, Y.; Magnusson, K. E. G.; Jalden, J. et al., Objective Comparison of Particle Tracking Methods. Nat Meth 2014, 11, 281-289. (35) Backlund, M. P.; Lew, M. D.; Backer, A. S.; Sahl, S. J.; Moerner, W. E., The Role of Molecular Dipole Orientation in Single-Molecule Fluorescence Microscopy and Implications for Super-Resolution Imaging. ChemPhysChem 2014, 15, 587-599. (36) Ortega Arroyo, J.; Cole, D.; Kukura, P., Interferometric Scattering Microscopy and Its Combination with Single-Molecule Fluorescence Imaging. Nat. Protocols 2016, 11, 617-633. (37) Sbalzarini, I. F.; Koumoutsakos, P., Feature Point Tracking and Trajectory Analysis for Video Imaging in Cell Biology. J. Struct. Biol. 2005, 151, 182-195. (38) Bobroff, N., Position Measurement with a Resolution and Noise-Limited Instrument. Rev. Sci. Instrum. 1986, 57, 1152-1157.

17


ACS Photonics

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Figure 1. Proper background correction reveals weak signal in scattering-based imaging. (a) Raw image, (b) background image, and (c) signal image. All displayed images are normalized by their mean intensities. The signal, whose position is indicated by the yellow arrow, is barely seen in the raw image (a). If the background image (b) is available, one can remove it from the raw image and the signal becomes clearly visible as shown in (c). Note that the range of colormap of (c) is different from that of (a) and (b).

18


Page 18 of 25

Page 19 of 25

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

ACS Photonics

Figure 2. Flowchart of the background estimation algorithm.

19


ACS Photonics

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Figure 3. Background extraction and correction for accurate localization. (a) Schematics of a particle moving on 11 x 11 grids in a stepwise, zig-zag manner. (b-e) Evaluation of the background estimation when the PSF is known (b,c) and unknown (d,e). The trajectory obtained by the true background serves as the reference, displayed both in (b) and (d) in yellow. We examine the trajectories obtained by the two estimated backgrounds where the PSF is known (b, blue) and unknown (d, red). Both estimated backgrounds give very accurate localizations (error < 1 nm rms). (c) and (e) show the error images of (b) and (d), respectively. The error image plots the difference between the corresponding localizations of the estimated trajectory and the reference trajectory. The color of the error plot represents the distance between true position and the center of the image. It appears that larger error occurs at the edge of the trajectory.

20


Page 20 of 25

Page 21 of 25

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

ACS Photonics

Figure 4. Monotonic decreasing and convergence of localization error over the iterative background estimation. The optimization process is about six times faster when the exact PSF is known. In both cases where the exact PSF is known and unknown, localization error (rms) less than 1 nm is achieved.

21


ACS Photonics

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Figure 5. The localization errors after background correction and the required number of iterations as a function of SNR (in the raw image). In (a), the localization error refers to the difference between the reconstructed trajectory and the exact pre-defined motion (without any source of noise). The dependence of the error on the SNR essentially reflects the effects of photon noise on localization, which is present in the reconstructed trajectory but absent in the pre-defined motion. In (b), the localization error refers to the difference between the two reconstructed trajectories where the true background and the estimated background are used for correction. The exact photon noise influences the localization in both cases. Therefore their localizations would be very the same if the two estimated backgrounds are similar, which is indeed the case shown here. (c) The number of iterations required for estimating the background at various SNR.

22


Page 22 of 25

Page 23 of 25

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

ACS Photonics

Figure 6. Experimental verification of background extraction and correction for accurate localization. (a) Raw, estimated background, and signal (normalized) images of 40 nm gold nanoparticle. (b-d) Results of a stepwise motion of 40 nm gold nanoparticle. (b) Trajectories obtained by background correction (measured background was used, yellow) and no correction (direct localization in the raw images, light blue). Due to the heterogeneous background, localizations without correction appear distorted. (c) Trajectories obtained after background correction by using the measured background (yellow) and the estimated background (red). Both trajectories reveal orthogonal, stepwise movements. (d) The error images of (b) and (c), plotted in light blue and red, respectively. It shows that the error is greatly reduced after background correction. (e-g) Results of a random walk of 40 nm gold nanoparticle. Similar to (b-d), we examined the background correction for particle motion of random walk. Again, accurate and precise localization was achieved by background correction with the estimated background. 23


ACS Photonics

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Figure 7. Experimental verification of background extraction and correction for tracking single virus particle on the plasma membrane of a live cell. (a) Raw, estimated background, and signal (normalized) images of single vaccinia virus particle on cell surface. (b-d) Results of the diffusive motion of the virus particle. (b) Trajectories obtained by background correction (measured background was used, yellow) and no correction (direct localization in the raw images, light blue). These two trajectories deviate from each other because of the bias in localization caused by the heterogeneous background. (c) Trajectories obtained by background correction where the measured background (yellow) and the estimated background (red) were used. (d) Error images of (b) and (c), plotted in light blue and red, respectively.

24


Page 24 of 25

Page 25 of 25

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

ACS Photonics

For Table of Contents Only

Manuscript title: Background Estimation and Correction for High-Precision Localization Microscopy Authors: Ching-Ya Cheng and Chia-Lung Hsieh Brief synopsis: We demonstrate a new method for estimating and correcting the heterogeneous but stationary background of a time series of microscope images by which the detection sensitivity and localization accuracy of single light emitter are greatly enhanced.

25


Background Estimation and Correction for High-Precision Localization

Recommend Documents