Research article Reliable State Identification and State Transition

Subscriber access provided by Kaohsiung Medical University

B: Biophysics; Physical Chemistry of Biological Systems and Biomolecules

Reliable State Identification and State Transition Detection in Fluorescence Intensity-Based Single-Molecule FRET Data Mélodie C. A. S. Hadzic, Richard Börner, Sebastian L. B. König, Danny Kowerko, and Roland K. O. Sigel J. Phys. Chem. B, Just Accepted Manuscript • DOI: 10.1021/acs.jpcb.7b12483 • Publication Date (Web): 08 May 2018 Downloaded from http://pubs.acs.org on May 10, 2018

Just Accepted “Just Accepted” manuscripts have been peer-reviewed and accepted for publication. They are posted online prior to technical editing, formatting for publication and author proofing. The American Chemical Society provides “Just Accepted” as a service to the research community to expedite the dissemination of scientific material as soon as possible after acceptance. “Just Accepted” manuscripts appear in full in PDF format accompanied by an HTML abstract. “Just Accepted” manuscripts have been fully peer reviewed, but should not be considered the official version of record. They are citable by the Digital Object Identifier (DOI®). “Just Accepted” is an optional service offered to authors. Therefore, the “Just Accepted” Web site may not include all articles that will be published in the journal. After a manuscript is technically edited and formatted, it will be removed from the “Just Accepted” Web site and published as an ASAP article. Note that technical editing may introduce minor changes to the manuscript text and/or graphics which could affect content, and all legal disclaimers and ethical guidelines that apply to the journal pertain. ACS cannot be held responsible for errors or consequences arising from the use of information contained in these “Just Accepted” manuscripts.

is published by the American Chemical Society. 1155 Sixteenth Street N.W., Washington, DC 20036 Published by American Chemical Society. Copyright © American Chemical Society. However, no copyright claim is made to original U.S. Government works, or works produced by employees of any Commonwealth realm Crown government in the course of their duties.

Page 1 of 46 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

The Journal of Physical Chemistry

Research article Reliable State Identification and State Transition Detection in Fluorescence Intensity-Based SingleMolecule FRET Data

Mélodie C. A. S. Hadzic†, Richard Börner†,*, Sebastian L. B. König‡, Danny Kowerko§, and Roland K. O. Sigel†,*

* Corresponding authors: [email protected], [email protected] † Department of Chemistry, University of Zurich, Winterthurerstrasse 190, 8057 Zurich, Switzerland § Department of Computer Science, Chemnitz University of Technology, 09111 Chemnitz, Germany ‡ Department of Biochemistry, University of Zurich, Winterthurerstrasse 190, 8057 Zurich, Switzerland

1 ACS Paragon Plus Environment

The Journal of Physical Chemistry 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Abstract: Single-molecule Förster resonance energy transfer (smFRET) is a powerful technique to probe biomolecular structure and dynamics. A popular implementation of smFRET consists in recording fluorescence intensity time traces of surface-immobilized, chromophore-tagged molecules. This approach generates large and complex datasets, the analysis of which is to date not standardized. Here we address a key challenge in smFRET data analysis: the generation of thermodynamic and kinetic models that describe with statistical rigor the behavior of FRET trajectories recorded from surface-tethered biomolecules in terms of the number of FRET states, the corresponding mean FRET values and the kinetic rates at which they interconvert. For this purpose, we first perform Monte-Carlo simulations to generate single-molecule FRET trajectories, in which a relevant space of experimental parameters is explored. Then, we provide an account on current strategies to achieve such model selection, as well as a quantitative assessment of their performances. Specifically, we evaluate the performance of each algorithm (CPA, STaSI, HaMMy, vbFRET, ebFRET) with respect to accuracy, reproducibility and computing time, which yields a range of algorithm-specific referential benchmarks for various data qualities. Data simulation and analysis were performed with our MATLAB-based Multifunctional Analysis Software for Handling smFRET data (MASH-FRET).


Page 2 of 46

Page 3 of 46 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


1. Introduction Since the initial proof of concept, single-molecule fluorescence techniques have become powerful tools to probe biomolecular structure and conformational dynamics.1–3 Single molecule Förster Resonance Energy Transfer (smFRET) uses the distance-dependent energy transfer between a donor and an acceptor fluorescent label to probe interdye distances between 2 and 10 nm.4 Spectral separation of donor and acceptor photon emission upon selective excitation of the donor allows to calculate the apparent transfer efficiency over time, which corresponds to the fraction of photons emitted by the acceptor.5 If required, absolute transfer efficiencies are generated upon applying a number of corrections to yield interdye distances according to Förster´s theory.6 Observation of single molecules over an extended period of time is usually achieved by anchoring them to the surface of a microscope slide, an approach that permits to resolve dynamics occurring on a timescale of milliseconds to hours.7,8 Two detection techniques are typically employed in surface-immobilized experiments: (i) the use of photon-counting avalanche photodiodes to record the arrival times of fluorescent photons, and (ii) the use of charge-coupled device (CCD) cameras that detect the number of photon arrivals within a fixed time window, usually millisecond-to-second.9 Both camera-based detection and time-binned photon counting data result in fluorescence intensity trajectories. If the molecular system under study adopts discrete conformations that are characterized by distinct interdye distances fluorescence intensity trajectories will feature multiple stable emission levels (states). A conformational change will engender an abrupt state transition, as conformational changes occur on a timescale considerably faster than the time resolution of the CCD camera and the typical binning interval used analyzing single-photon counting data.10 Such fluorescence intensity trajectories contain, in principle, exhaustive 3 ACS Paragon Plus Environment


information about state-specific mean interdye distances3,11 and/or dye local environment,12–14 the equilibrium configuration of states and the rates at which they interconvert. Retrieving this information from the raw data, however, relies on careful selection of a model that accurately describes the thermodynamic and kinetic behavior. In the context of a camera-based time-binned smFRET experiment, model selection entails the determination of (i) the number of states J hidden in the noise, (ii) the corresponding mean FRETj values, (iii) the relative occurrence of states popj , and (iv) the kinetic rate coefficients kjj’ associated with state-to-state transitions that are enumerated up to J(J-1).15,16 Model selection can be accomplished in numerous ways. At the simplest level, the number of states can be deduced from the way the experiment was designed, for example when cationmediated RNA–RNA interactions only show two distinct states.16 Similarly, the number of FRET means can be determined heuristically from the number of apparent peaks present in the overall distribution of FRET data.17 For complicated molecular systems, high-resolution structures and computational studies are often needed in interpreting trajectories.18–20 However, structural data is often not available and transient conformational states along a folding pathway usually cannot be isolated for subsequent structural elucidation.21 For many biomolecular systems, the entire folding pathway is in fact not known.22,23 In those cases, statistical model selection is required. Statistical modeling of an experimental fluorescence intensity trajectory as a Markov chain hidden in experimental noise (hidden Markov model, HMM), has been pioneered by McKinney and co-workers.24 In their software called HaMMy, the parameters of the model (state values, noise width and transition probabilities) are varied to maximize the likelihood (ML) of reproducing the specific experimental data set. Nowadays, a number of freely available software packages employ HMM to analyze smFRET data (see Table 1).17,25–32 HaMMy is the earliest 4 ACS Paragon Plus Environment

Page 4 of 46

Page 5 of 46 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


standalone software dedicated to the discretization of single molecule FRET trajectories.27 HMMs are inferred for single trajectories and the overall model is determined by fitting transition density plots (TDP) with a mixture of two-dimensional Gaussians, followed by selecting the mixture that renders the lowest Bayesian information criterion (BIC). Because the likelihood monotonically increases with the complexity of the model, ML-based methods tend to overestimate the complexity of the model. To overcome this limitation, other algorithms perform hidden Markov modelling by maximizing the evidence (ME) instead of the likelihood. Here, the evidence denotes the likelihood averaged over all possible model parameters defined for a particular model complexity. vbFRET

28

and ebFRET

30,33

are two MATLAB-based tools that

apply ME-based selection using single trajectories (vbFRET) or the whole data set (ebFRET), respectively. Another way to overcome ML overfitting is to select the lowest BIC estimate for single trajectories or for an ensemble set of trajectories, an approach implemented in the software SMART29 or postFRET17 respectively. Other discretization methods do not rely on HMMs. For example, STaSI is a MATLAB-based tool that uses the student's t-test to identify state transitions and the minimum description length (MDL) to select the optimum number of states.34 In addition, change-point analysis (CPA) is one of the most straightforward step-detection strategies used for single trajectory discretization, as transitions are identified when the change in signal exceeds a certain threshold amplitude. Other pure step-detection methods include the Velocity threshold,35 χ2-minmisation,36 both applied to step-like optical tweezer data, and the Derivative of Gaussian wavelet.37 The performances of these step-detection algorithms on simulated Poisson stepper data were previously reported in a comparative study.38 In contrast to HMM-based approaches, neither STaSI nor step-detection methods assume an exponential distribution of state lifetimes and are thus model-free. 5 ACS Paragon Plus Environment


Several factors are known to complicate statistical model selection: (i) A rugged free energy landscape. Numerous biomolecules stably adopt several conformations, resulting in multiple, often overlapping FRET efficiency states.39 (ii) Kinetics on multiple timescales. Allegedly simple molecular systems like RNA enzymes, DNA G-quadruplexes or RNA-binding proteins are often able to undergo multiple conformational changes occurring on different timescales.40–42 Fast and slow processes can often not be simultaneously investigated, as the high photon emission rates required for studying fast kinetics implies fast fluorophore photobleaching, which prevents the observation of long-lived states. (iii) Intrinsically low signal-to-noise ratio (SNR). As smFRET is based on the detection of single fluorophores, the SNR is inherently low. The noisier the trajectories, the broader the distribution of FRET values for a single state, which increases the overlap between state distributions. Multiple states with unresolvably close or identical FRET efficiencies lead to degenerate FRET efficiencies. Degeneracy may result in multi-exponential kinetics and/or continuous distribution in state values, both of which can considerably complicate model selection.43–45 (iv) Organic fluorophores do typically not behave like a robust "light bulb".46 Instead, they often display intensity fluctuations induced by photophysical processes such as triplet blinking, redox blinking, and photochromic blinking. In addition, intensity fluctuations can have stereochemical origin. For example, two diastereomers co-exist for the widely used class of unlocked cyanine dyes, where the trans-isoform is known to be fluorescent, whereas the cis-isoform is not.47 The molecular environment of the fluorophore can shift the equilibrium of the cis-trans isomerization, and hence, alter its emission intensities, quantum yields, and fluorescence lifetime.48 (v) Intermolecular differences in fluorescence intensity and FRET values, so called heterogeneous broadening induced by instrumental or molecular sources, is frequently encountered in single molecule measurements and make the identification of a


Page 6 of 46

Page 7 of 46 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


global model more difficult.27 Most of the above-mentioned model selection methods were published along with an evaluation of their performances on synthetic and/or experimental data.28,30,34,49 However, an independent comparative study of different strategies for statistical model selection in the context of single molecule data analysis has, to the best of our knowledge, not been reported. This article assesses the performances of representative model selection algorithms using simulated data. In these simulations, we considered the above-mentioned factors complicating statistical model selection using experimental data (see (i) – (v), previous paragraph). Specifically, we varied the trajectory length, the SNR, and the transition rate coefficients. In addition, we accounted for heterogeneous broadening. These simulated data were then used to assess five model selection algorithms: the stand-alone program HaMMy, the MATLAB-based scripts vbFRET, ebFRET and STaSI, as well as a home-written implementation of the change-point analysis algorithm (CPA) (Table 1). CPA is a pure step-detection method, i.e., it does not group states. It was therefore used to exemplify lacking the grouping of states in model selection. Each method tested was scored based on the accuracy of the restored model, the reproducibility of the result and the computation cost. Finally, we also evaluated the ability of each algorithm to identify the correct number of states and their corresponding mean FRET. Table 1. Overview on the model selection algorithms tested. Method

Type and origin

parameters

source

HaMMy

Standalone HMM selection on smFRET piecewise signal.

Maximum J, initial FRET centers.

Software HaMMy 27

vbFRET

Matlab script available with or without GUI. HMM selection on smFRET piecewise signal.

Maximum J, number of initialization. 7

ACS Paragon Plus Environment

Matlab script vbFRET 28,50


Page 8 of 46

ebFRET

Matlab script available with GUI. HMM selection on a trajectory pool.

Maximum J, number of initialization.

Software ebFRET 30,33

STaSI

Matlab script available with or without GUI.

Maximum J.

Matlab script STaSI 34

CPA

Home-written Matlab script

Tolerance level X

adapted from 51

2. Methods 2.1

Simulation

To test the performance of the five algorithms in terms of (i) their ability to correctly identify the number of states and (ii) to accurately restore kinetic rate constants, synthetic data sets were generated using a home-built Matlab-based software (Multifunctional Analysis Software for Handling of smFRET data; MASH-FRET) recently introduced in ref. 5 as SMV categories 5-7. Briefly, fluorescence intensity trajectories comprising Poisson-noise were generated, in which experimentally relevant parameters were varied, i.e., the SNR, the kinetic rate coefficients, the trace length, and molecule-to-molecule variations in total fluorescence intensity and FRET values. In general, each data set comprised 100 fluorescence intensity trajectories, for the simulation of which the following default settings were used: •

Trace length L = 4200 frames. This value roughly corresponds to the maximum size of a video recordable with the EMCCD camera Andor iXon ultra 897 used for comparison in camera noise modelling.

•

Total intensity Itot,0 = 34.6 photons per frame. This value corresponds to an SNR value of 3.5. In our hands, this SNR value is typically observed in an intensity-based smFRET experiment.

•

Kinetic model, number of states J0, their corresponding mean FRET values FRETj,,0, and 8 ACS Paragon Plus Environment

Page 9 of 46 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


rate coefficients. (i) For the assessment of an algorithm’s ability to identify the correct number of FRET states, traces were characterized by four FRET states (J0 = 4) featuring sequential transitions between them (A ↔ B ↔ C ↔ D). Mean FRET values FRETj,,0 were set to 0.3, 0.45, 0.6, and 0.8, respectively. Transition rate coefficients were set to k12 = k21 = 0.05 s–1, k23 = k32 = 0.13 s–1, k34 = k43 = 0.03 s–1, values inspired by an earlier experimental smFRET study.27 (ii) For the assessment of an algorithm’s ability to correctly restore rate coefficients, we simulated trajectories of a biomolecular system toggling between an open and a closed conformation (A ↔ B). Trajectories featured two states (J0 = 2) centered around 0.3 (open) and 0.7 (closed), respectively. Transition rate coefficients were set to kopening = 0.02 frame–1 and kclosing,0 = 0.05 frame–1. •

Intermolecular heterogeneity concerning FRETj,0 and Itot,0 was assumed to be absent. Hence, wFRET = 0 and wtot,0 = 0.

•

Instrumental imperfections, such as bleed through and direct excitation as well as detection efficiency and fluorophore quantum yields, usually referred to as gamma correction are taken into account as previously described.5 Fluorescence background was omitted in our simulations. Both, instrumental imperfections and fluorescence background, are usually corrected for in single-molecule fluorescence experiments. However, the signal-to-background ratio is mutually linked to the SNR, and has to be considered when comparing experimental results with simulations used herein.

For each data set, only one simulation parameter was varied at a time, fixing the other to default values. (i) To evaluate the ability of all algorithm to correctly identify the number of states, the total fluorescence intensity was varied (7 ≤ Itot,0 ≤ 97 photons per time bin or frame), 9 ACS Paragon Plus Environment


corresponding to SNR values ranging from 1 to 7. (ii) To evaluate the ability to restore transition rate coefficients, we varied the trajectory length (70 frames ≤ L ≤ 64'000 frames), the SNR (1 ≤ SNR ≤ 7), the fast transition rate (0.02 ≤ kclosing,0 ≤ 20 frame–1) and the cross-sample heterogeneity with regard to FRETj,0 and Itot,0. Heterogeneity associated with FRETj,0 was modeled with two Gaussian probability density functions (PDFs) centered around FRET1,,0 = 0.3 and FRET2,,0 = 0.7. Each PDF was characterized by a specific standard deviation 0 ≤ wFRET ≤ 0.15, i.e., 0 to 2.3 times the standard value suggested previously (wFRET = 0.065).27 Heterogeneity associated with Itot,0 was modeled as a Gaussian distribution centered at Itot,0. The standard deviation wtot,0 was set to 0 ≤ wtot,0 ≤ 0.5 x Itot,0. Finally, camera noise was considered in all simulated trajectories. In order to realistically model camera noise, the experimental distribution of readout noise and spurious charge noise induced by clock-induced charges (CIC) of an EM-CCD was determined from an SMV recorded with closed camera shutter (data shown elsewhere).5 The signal intensity distribution was then fitted to the NExpN camera noise model proposed in ref. 5 and further used to account for camera noise. 2.2

Evaluation procedures

To evaluate the ability of different algorithms to correctly identify the number of states, fluorescence intensity trajectories were analyzed as illustrated in Figure 1A. Briefly, the following steps were performed: (1) Fluorescence intensity trajectories were converted to FRET traces,5 which were then analyzed using CPA, STaSI, vbFRET and HaMMy, all of which are implemented in MASH-FRET. Please note that each simulated data set consisted of 100 fluorescence intensity trajectories. Hence, we obtained 100 inferred state trajectories per data set per algorithm. (2) State trajectories were randomized via bootstrapping,52 yielding 100 bootstrap samples, each composed of 100 state trajectories.16 (3) TDPs were generated from bootstrap 10 ACS Paragon Plus Environment

Page 10 of 46

Page 11 of 46 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


samples, in which FRET values before and after transitions are represented as a heat map. Subsequently, TDPs were Gaussian-convoluted (Section S2 of the Supporting Information). (4) Gaussian-convoluted TDPs were fitted by J×(J-1) Gaussian mixture models with 1 ≤ J ≤ 10 states. Here, the optimum number of states and the corresponding FRETj values were obtained via ML-BIC. As 100 bootstrap samples were used, we obtained 100 estimates of J denoted Jm (see also: Section S2 of the Supporting Information).16 (5) Finally, the mean number of inferred states J̄ and the corresponding standard deviation σJ were calculated from Jm. In the case of ebFRET, model selection was performed according to the protocol provided by the authors,23,31 yielding only one model estimate per data set. Thus, the statistical error could not be determined (see also: Section S2.1.4 of the Supporting Information). To evaluate the ability to restore transition rate coefficients, a similar approach was used (Figure 1B): (1-2) as before. Importantly, all algorithms were constrained to detect two states in order to exclude erroneously detected/missed transitions due to an incorrect kinetic model. (3) Dwell time analysis was performed to extract the rate coefficients kopening and kclosing.46 For this purpose, dwell times, i.e., the time spent in one state before transitioning to another, were extracted from state trajectories. Subsequently, all dwell times associated with transitions from 0.3 to 0.7 (closing) and 0.7 to 0.3 (opening), respectively, were binned to cumulative histograms.53 These dwell time histograms were fitted to monoexponential decay models, yielding a total of 100 estimates of the transition rate coefficients ki,m associated with opening and closing. (4) Mean apparent rate coefficients k̅i and the corresponding standard deviations σk,i were calculated from ki,m and converted into a deviation ∆ki from the true simulated rate ki,0: ∆ k i = 100 ×

( ki − ki ,0 ) ki ,0

and

σ ∆k ,i = 100 ×

3 ⋅ σ k ,i ki , 0

(1)

In the particular cases of ebFRET and HaMMy, rate coefficients and error intervals σ∆k,i were 11 ACS Paragon Plus Environment


collected from the output files and deviations were calculated according to Eq. (1). Finally, average computation times were quantified by processing 100 FRET trajectories adopting 1 to 10 states. Data analyses were performed using MATLAB R2011a on a desktop computer equipped with an Intel Core i5-2320 processor (3.00 GHz clock speed). 2.3

Quantification of method accuracy

To quantify the accuracy of the evaluation procedures described above, we used a classification approach commonly performed in data mining and information theory and recently adapted in the field of super resolution microscopy and single molecule analysis.5,54 Specifically, we compared the outcome of each evaluation procedure, i.e., the inferred result, to the original simulated data (ground truth, GT). Hence, the inferred results were classified as true positives (TPs), false positives (FPs), and false negatives (FNs). Detection algorithms were not tested for true negatives (TN), thus, we set TN = 0. In the context of the evaluation of an algorithm’s ability to identify the correct number of states and their corresponding mean FRET values, transition state models inferred from Gaussianconvoluted TDPs were classified as TPs, FPs, or FNs according to the following criteria (Figure 1C): •

TP: The inferred number of states Jm matches the true number of states J known form the simulation parameters, i.e., Jm = J = 4. In addition, inferred mean FRET values FRETj,m agree with the input parameters within a tolerance range ± 0.01 FRET units.

•

FP: Jm = J = 4. However, the inferred mean FRET values FRETj,m do not consistently agree with the input parameters within a tolerance range ± 0.01 FRET units.

•

FN: The inferred number of states does not match the true number of states, i.e., Jm ≠ J = 12 ACS Paragon Plus Environment

Page 12 of 46

Page 13 of 46 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


4. In order to assess the ability of an algorithm to accurately detect state transitions, inferred state trajectories were classified in a similar fashion (Figure 1D): •

TP: The inferred transition matches the true transition known form the simulated state trajectory within a tolerance range of ± 1 frame.

•

FP: Detected transitions without true transition known form the simulated state trajectory.

•

FN: No detected transitions with true transition known from the simulated state trajectory.

Subsequently, we calculated the algorithm-specific accuracy, precision and recall associated with model selection and step detection as described in ref. 5 using Eqs. (2)-(4): accuracy =

TP TP + FN + FP

(2)

precision =

TP (3) TP + FP

recall =

TP TP + FN

(4)

In the particular case of ebFRET, only one model was obtained. Hence, the output was either classified as success (accuracy = 1, recall = 1, precision = 1) or error (accuracy = 0, recall = 0, precision = 0). 3. Results and Discussion 3.1

State identification

The number of FRET states J and the corresponding FRETj values describe the folding and/or interacting pathway of a molecular system studied with smFRET. Correct identification of these parameters provides a basic description of the system with regard to introduced reaction coordinates, i.e. the distance between respective donor and acceptor positions.55 To highlight the strengths and limitations of five algorithms in this context, we evaluated their ability to identify the correct number of states and their corresponding mean FRET values. In addition, we also



determined the computation cost and the accuracy , precision and recall of each model generated (see Eqs. (2)-(4)). The SNR of trajectories was varied to assess the impact of noise on the quality of the inferred models and to determine the noise level at which the methods become inaccurate (Figure 2). At high signal-to-noise ratios (SNR = 5, 7), CPA, STaSI, vbFRET, and HaMMy all faithfully reproduce the true number of states J0 = 4, with STaSI (SNR = 5) and vbFRET (SNR = 7) inferring the exact number (J̄ = 4.0 ± 0.0, Figure 2A). In addition, models inferred by vbFRET, STaSI, and HaMMy are characterized by high accuracy , perfect precision and almost perfect

recall , whereas the accuracy of FRET values in CPA models is close to zero (Figure 2B). Low accuracy is an intrinsic limitation of CPA, which does not group states prior TDP fitting (Figure 3A). Transitions between this gradient of FRET-states result in a continuous smear in the corresponding TDPs (Figure S1). This smear leads to errors in the BIC-based model selection, favoring models that include additional intermediate states (FNs). With regard to ebFRET, an artefactual fifth FRET state is erroneously identified at 3 ≤ SNR ≤ 7, leading to accuracy = 0. This additional state is characterized by an almost identical FRET value as another true state, which defines them as degenerated. Systematic overestimation of the number of states has already been observed for ebFRET when was applied to experimental data.33 The incidence of additional states has been attributed to statistically significant data variations that are not assigned to molecular conformation, such as transition averaging or fluorophore blinking. Our results show that ebFRET may also overestimate the number of states in simulated trajectories. This deviation from earlier observations could be due to the way camera noise was modeled in our simulation, i.e., taking into account clock-induced charges translation into an exponentially distributed signal distribution.5,56 Interestingly, we found the total evidence lower bound to level 14 ACS Paragon Plus Environment

Page 14 of 46

Page 15 of 46 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


off for J ≥ J0 and SNR ≥ 3 (Figure S3), which raises the interesting possibility to further refine the model criteria of ebFRET to avoid the identification of artefactual states. A detailed description of model selection with ebFRET is given in Section S2.1.4 of the Supporting Information. Setting the SNR to values that are, in our hands, typically observed in a camera-based smFRET experiment (SNR = 4), only STaSI and vbFRET successfully identify the true number of states (Figure 2A). Here, STaSI reaches a high accuracy = 0.74, whereas the accuracy of FRETj values inferred by vbFRET reaches only 0.37 (Figure 2B). This result is in accordance with past evaluations, in which STaSI has been reported to faithfully identify the FRETj values, whereas vbFRET missed real states and erroneously detected "blur states". The difference in performance between STaSI and vbFRET has been attributed to the use of different fitting penalties, which is more sophistically defined for STaSI, as it includes more descriptive parameters (see Eq S1 in the Supporting Information). In contrast, the number of states and FRETj values inferred using HaMMy are biased by an increasing FN number, though, accurate estimations are obtained at higher SNR (SNR = 5). Close inspection of TDPs (Figure S1) reveals (i) probability clusters centered around the FRETafter = FRETbefore diagonal for vbFRET, which correspond to transitions shown in the trajectory in Figure 3B, and (ii) clusters at extreme FRET values for vbFRET and HaMMy, which correspond to transitions shown in trajectories of Figure 3C and D. It has previously been reported that vbFRET tends to discretize the noise around a detected state, which leads to (i).16,28,34,49 Both, vbFRET and HaMMy are increasingly susceptible to detecting artefactual excursions to extreme FRET values when clock-induced charge (CIC) noise is included in the simulations. As one of the assumptions made by HaMMy and vbFRET is normally distributed 15 ACS Paragon Plus Environment


noise, the CIC contribution may introduce analytical artefacts. We tested whether this had an influence on vbFRET performances (see Section S2.1.5 of the Supporting Information). In conclusion, although an exponential-tailed Gaussian noise can introduce changes in FRET values leading to artefactual states, it has no effect on the performance of vbFRET. A possible explanation is that the Poisson noise used in our simulations and elsewhere5,57 may affect the symmetry of the data distribution at low SNR values, and hence, influences HMM-based method outcome.33 For very low signal-to-noise ratios (SNR < 3), all algorithms are prone to either under- or overestimating the complexity of the model, which leads to a decrease in recall (Figure 2B). Even though the correct number of states may be identified (STaSI, ebFRET, SNR = 2; HaMMy, SNR = 1, 3), the corresponding inferred FRETj values are subject to large errors ( precision = 0). The computation times per trajectory required by all five algorithms are shown in Figure 2C. Here, the true number of states J0 was increased from 1 to 10. In the case of STaSI and CPA, the computation time per trajectory is independent of the number of states. In contrast, the time required by HaMMy, vbFRET and ebFRET increases exponentially with J0. When complex time traces are analyzed (J0 ≥ 4), STaSI outperforms the other methods with a computation time of 1.5 ± 0.3 s per trajectory (L = 4200 frames), followed by CPA (3.4 ± 0.6 s), HaMMy (5.2 ± 0.8 s), vbFRET (15.0 ± 2.9 s) and ebFRET (16.3 s). In summary, our results demonstrate that STaSI used in conjunction with TDP fitting is most robust and accurate with regard to model selection at high SNR and outperforms the other algorithms assessed in terms of computation time. On the other hand, CPA and TDP fitting turn out to be non-ideal for model selection. With this approach, none of the methods successfully 16 ACS Paragon Plus Environment

Page 16 of 46

Page 17 of 46 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


resolve the four states hidden in trajectory at SNR ≤ 3. We defined a minimum SNRlim for each method, corresponding to the lowest SNR for which the calculated accuracy is greater than 0.5: SNRlim = 4 for STaSI and SNRlim = 5 for vbFRET and HaMMy. Unfortunately, SNRlim could not be determined for ebFRET and CPA, as the accuracy is zero for all SNR tested here. In particular, for SNR > 2, ebFRET infers five-states models with two states being degenerated in their FRET values, leading to a model accuracy of zero (Section S2.1.4 of the Supporting Information). 3.2 3.2.1

Step detection Influence of the trajectory length

The number of data points in a trajectory is limited either by the photobleaching of dyes, which prevents further calculation of FRET, or by the acquisition software that constrains the size of the recorded video. Early photobleaching of fluorophores is typically observed when high excitation intensities are used. In turn, the number of frames in a video can be extended if the spatial resolution can be reduced. Here, we evaluate the impact of the trajectory length (70 ≤ L ≤ 64'000 frames) on the ability of step-finding algorithms to correctly determine the transition rate coefficients that characterize the interconversion between two FRET states (Figure 4, first column). In general, short trajectories (70 ≤ L ≤ 140 frames) displayed only a low number of transitions (1.4 ≤ Ntrs,slow ≤ 2.5). Hence, only a low number of dwell-times was used to build dwell-time histograms, which leads to large standard deviations σk,i associated with apparent rates k̅i (Figure 4). Under these conditions, vbFRET and ebFRET display the highest detection accuracy, i.e.,

accuracy = 0.92 and 0.93 respectively (Figure 4C). With an accuracy below 0.5, HaMMy 17 ACS Paragon Plus Environment


Page 18 of 46

is not suitable for very short trajectories (L ≤ 70 frames): the algorithm generates a significant number of FPs, leading to a reduced precision (Figure 4C) and an overestimation of the opening rate (Figure 4A). However, the number of FPs detected by HaMMy drops upon increasing the trace length to 350 frames or more, resulting in a accuracy value of 0.96. In contrast, the

accuracy associated with CPA consistently remains average (≤ 0.73). Interestingly, kopening is overestimated at L = 140 frames when traces are analyzed with CPA, STaSI or vbFRET (Figure 4A) without any significant concomitant change in

accuracy

(Figure 4C). As kclosing is not affected (Figure 4B), we believe that this effect is intrinsic to the chosen dwell-time selection, which consists in discarding the first and last dwell-times of each inferred state trajectory, because their duration is unknown. However, this approach leads to an overrepresentation of short dwell-times in trajectories featuring few transitions, as long dwelltimes are the more likely to be discarded. Nonetheless, including discarded dwell-times does not merely improve the estimation of kopening (Section S2.2.2 of the Supporting Information). As the trace length increases, and thus the average number of transitions per trace, apparent rate coefficients determined by HMM-based methods (HaMMy, vbFRET and ebFRET) converge to the ground truth. In the specific case of HaMMy, the loss of transition detection accuracy when analyzing very long trajectories (L = 64'000 frames) is due to the limited size of the outcome sequence fixed at 49'999 data points or less (Figure 4C). As a consequence, all transitions occurring within the last 14'001 frames were not detected, resulting in an increased number of FNs and a decrease of detection recall (Figure 4C). As the trace length increases, the error ranges inferred by ebFRET and vbFRET converge to solely a few percent. When the maximum trace length is reached (L ≥ 16'000 and 64'000 frames respectively), the two algorithms fail to faithfully restore the opening transition rate and, instead, underestimate its 18 ACS Paragon Plus Environment

Page 19 of 46 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


value. STaSI and CPA, however, systematically underestimate both rates even when very long trajectories are analyzed. The reason for this underestimation is that model-free approaches are more prone to skip short-lived states, as illustrated in Figure S4, which increases the number of FNs. Consequently, model-free step finding algorithms are characterized by recall values that are systematically lower than those of their HMM-based counterparts (Figure 4C). These results are in accordance with a previous comparison of a HMM-based method with CPA, where the number of missed transitions has been reported to be 16 times greater in the case of CPA.39 Taken together, overall performance in detecting transitions as a function of trace length follows the order ebFRET > vbFRET > HaMMy > STaSI > CPA. The probabilistic algorithms tested herein are more accurate in recapitulating transition rate coefficients for standard-to-long trajectories than model-free ones. Remarkably, ebFRET and vbFRET are most accurate when very long trajectories are analyzed ( accuracy = 0.94 for 64'000 data points) and statistical errors are minimized at L ≥ 700 frames. Taking into account the average computation time required to process a two-state trajectory (Figure 2C), HaMMy appears to be a good compromise between accuracy and speed when standard length trajectories are analyzed, albeit at the expense of increased uncertainty. It should be noted, however, that the stand-alone program HaMMy does not discard the first and last dwell-times of a FRET state trajectory, which may bias the result (see above). Moreover, in the case of a trajectory displaying more than two states, the last dwelltime cannot be assigned to a specific forward transition, as the next state is unknown. 3.2.2

Influence of transition rates

The interconversion between two conformational states of a molecule, or a system, is governed by two transition rate coefficients. In general, an increase in rate coefficients results in a decrease of the average dwell-times and an increased number of transitions within one trajectory. This also 19 ACS Paragon Plus Environment


implies that the incidence of dwell times characterized by a duration of less than a frame becomes more likely. Such brief excursions to a FRET state appear as transitions to intermediate FRET values that are linear combinations of the two FRET values involved in the state transition. Short dwell-times and time-averaging of FRET data are known to render the detection of state transitions daunting. Here, we evaluate the performances of the five step-detection algorithms upon variation of one rate coefficient, kclosing (Figure 4, second column). For kclosing,0 = 0.04 frame–1 and higher, data processing with STaSI and CPA leads to systematic deviations from the true fast rate, corroborating that model-free step finding algorithms are prone to missing short-lived states (Figure S4). The significant number of FNs translates into comparably low

accuracy and low recall

values (Figure 4C). In turn, HMM-based

approaches are more robust. Indeed, vbFRET and ebFRET are able to correctly restore transition rates up to kclosing,0 = 0.04 frame–1 with high

accuracy . HaMMy is the only method that

successfully recapitulates the two apparent rate coefficients (∆kopening = –6.5 ± 9.9% and ∆kclosing = –7.9 ± 9.8%) at kclosing,0 = 0.1 frame–1. Increasing kclosing,0 generates dwell-times of one frame and shorter with a probability P ≥ 0.63 at kclosing,0 ≥ 1 frame–1. Under these conditions, all algorithms fail to restore the true rate coefficients. Remarkably, HMM-based algorithms maintain a very high recall at kclosing,0 = 20 frame–1, i.e., most transitions were correctly detected. It is a decrease in detection precision , i.e., an increasing FP number, that further leads to a drop in

accuracy . Therefore, our results demonstrate that probabilistic methods are prone to consider noise spikes as state transitions if the mean state life time is less than 1 frame. Taken together, HaMMy is the most performant algorithm with regard to restoring fast rate coefficients (kclosing,0 ≤ 0.1 frame–1). Although vbFRET achieves very similar accuracy and a high resolution limit for the fast rate (kclosing,0 = 1 frame–1, Figure 4B), which is in accordance 20 ACS Paragon Plus Environment

Page 20 of 46

Page 21 of 46 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


with past evaluations,28,50 ∆kopening,0 increases dramatically under the same conditions (∆kclosing ± σopening = 31.2 ± 3.8%). Finally, the two model-free approaches CPA and STaSI are shown to be prone to missing short dwell-times. They are thus not suitable for studying fast kinetics relative to the frame rate of the CCD camera, in particular 1/k < 25 frames and smaller (Figure 4, second column). 3.2.3

Influence of the signal-to-noise ratio

The noise width of a FRET trajectory is defined by the single molecule fluorescence intensity, the background contribution and the characteristics of the detector. A large noise width, or a low SNR, indubitably increases the overlap between FRET levels, which makes the distinction between states more complicated. Additionally, noise spikes emerging from CIC may be identified as state transitions by step-finding algorithms, which biases the calculation of the transition rate coefficients. To determine the accuracy of step-finding algorithms as a function of trajectory noise, we inferred rate coefficients and the step detection accuracy using trajectories characterized by 1 ≤ SNR ≤ 7 (Figure 4, third column). As discussed in the previous section, CPA and STaSI fail to restore rate coefficients at all SNRs assessed, because short dwell-times are consistently missed. In turn, apparent rate coefficients calculated from state trajectories inferred by HaMMy, vbFRET and ebFRET are in good agreement with the ground truth at SNR = 7, albeit deviations are observed for SNR < 7. Analogous to the approach outlined in Section 3.2.2, we identified a minimum signal-to-noise ratio SNRlim, at which a specific algorithm succeeded in correctly restoring both rate coefficients. For vbFRET and HaMMy, SNRlim = 2. In contrast, ebFRET achieves a SNRlim value of 3, because it slightly overestimates the slow rate (∆kopening ± σopening = 4.0 ± 3.3%) at SNR = 2 (Figure 4A). As a caveat, it should be noted that the accuracy associated with detecting transitions at SNR = 21 ACS Paragon Plus Environment


2 is much lower for HaMMy than for vbFRET and ebFRET. As mentioned above, ML approaches such as HaMMy tend to fit experimental noise, resulting in a higher number of FPs in comparison to the ME-based algorithms assessed herein. Consequently, HaMMy also achieves a lower

precision value than both ME-based algorithms under these conditions (Figure 4C).

Despite the fact that both ebFRET and vbFRET are ME-based, ebFRET is much more accurate than vbFRET at SNR = 1, because ebFRET automatically excludes FRET efficiencies outside the boundaries of [-0.2 1.2]. (Section S3.3 of the Supporting Information). When these boundaries are applied prior analyzing data with vbFRET, the algorithm is able to restore both transition rates and scores an accuracy even higher than ebFRET (Figure S7A and S7C), which is in disagreement with past results.30 However, the synthetic trajectories used here are 42 times longer than the ones used in the earlier evaluation, which greatly influences the performances of algorithms. Indeed, we show that when trajectories are shortened down to 70-140 frames, ebFRET becomes the only algorithm that is able to restore correct transition rates. Therefore, it is possible that the comparatively good performance of ebFRET reported previously was in fact due to a rather low trajectory length. In conclusion, we ranked the performances of step-finding algorithms on noisy trajectories: ebFRET ≈ vbFRET > HaMMy > STaSI > CPA. 3.2.4

Influence of heterogeneous broadening in FRET

The molecule-to-molecule variations in FRETj values are commonly attributed to inaccurate correction of channel-specific intensity signals, mislocalization of donor and acceptor molecules or to differences in the molecular environment that induce fluorescence enhancement or inhibition. This type of fluctuations renders data analysis more complex, as a single conformational state is not anymore characterized by a single mean FRET value, but a distribution of mean FRET values. To characterize and quantify the effect of heterogeneous 22 ACS Paragon Plus Environment

Page 22 of 46

Page 23 of 46 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


FRET broadening on the ability of step-detection methods to determine rate coefficients, we processed FRET trajectories of various heterogeneity using HaMMy, vbFRET, ebFRET, STaSI and CPA. The level of FRET heterogeneity is given by the width wFRET of the state distribution across the trajectory set. As shown in Figure 4 (fourth column) a variation of wFRET does not have a significant influence on the apparent rate deviations ∆ki and the corresponding standard deviations σk,i. Very large wFRET values may lead to strong overlap between the two FRET species (Figure 5A, case a), albeit it is equally likely to increase the FRET spacing (Figure 5A, case b). We believe that the two effects counter-balance each other in the case of the five methods investigated in this study. It is possible, however, that molecule-to-molecule variations in FRET state values may have a greater impact on the outcome of an analysis where trajectories with more than two states are considered. To estimate the relative efficiency of algorithms, we quantified the number of trajectories that were erroneously excluded from rate calculations because no transition was identified by the step-finding algorithm (Figure 5B). As expected for trajectories with poorly spaced FRET states, the number of excluded molecules generally increases at larger wFRET. At wFRET = 0.15, HaMMybased analysis leads to the highest number of excluded trajectories (18 out of 100), while ebFRET does not discard any trajectory (0 out of 100, Figure 5C). Representative time traces are shown in Figure 5D. In fact, the accuracy of HaMMy steeply decreases from 0.94 to 0.66 upon increasing wFRET from 0 to 0.15. On the other hand, vbFRET’s accuracy decreases only from 0.95 to 0.85 within the same wFRET range. In summary, ebFRET and vbFRET are most accurate when molecule-to-molecule variations in FRETj values are present in the sample. In turn, ebFRET outperforms the other methods in terms 23 ACS Paragon Plus Environment


of robustness, as it does not exclude any trajectory from rate calculations. This result is not surprising, as it is the only algorithm tested here which models the variation of inferred parameters across the trajectories. It is important to note that these conclusions should not be generalized, as state overlap in this evaluation is solely due to the proximity of the FRET centers. Indeed, when state overlap is due to low SNR (see previous subsection), rate coefficients are under- or overestimated. 3.2.5

Influence of heterogeneous broadening in total fluorescence intensity

Similarly to FRET values associated with conformational states, the total fluorescence intensity may also vary from one molecule to another. In particular, an inhomogeneous excitation field or quantum yield variations induce such heterogeneity. As demonstrated in Section 3.2.3, the SNR of a trajectory directly impacts the outcome of step detection algorithms. Here, we evaluate how well step detection algorithms can correct for molecule-to-molecule variations in SNR by processing trajectories with different heterogeneity levels. These were defined as the width wtot of the total fluorescence distribution across the simulated trajectory set (Figure 6). In general, variation of the total fluorescence intensity has no significant effect on the inferred apparent rate coefficients at 0 ≤ wtot,0 ≤ 17.3 pc/frame (Figure 4, fifth column). Consistent with this observation, the number of apparently static trajectories is 0 for most conditions assessed (wtot,0 < 17.3 pc/frame, Figure 6C). Only when cross-sample variability increases to unrealistic levels (wtot,0 = 17.3 pc/frame, Figure 6B), HaMMy and ebFRET exclude two trajectories, CPA three, and five trajectories for STaSI and vbFRET. For HaMMy, STaSI and vbFRET, excluded trajectories display transitions that involve extreme FRET values (above 1.2 or below -0.2). Representative state sequences inferred under these conditions are shown in Figure 6D. In summary, molecule-to-molecule variations in fluorescence intensity do not affect the 24 ACS Paragon Plus Environment

Page 24 of 46

Page 25 of 46 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


performance of the step finding algorithms tested herein. However, ebFRET is the method that is recommended to use in case of SNR heterogeneity. In particular, the algorithm show greater robustness, as it includes the largest number of trajectories for estimation of transition rate coefficients. 4. Conclusions A crucial stage in the analysis of camera-based smFRET experiments is the identification of abrupt changes in a noisy signal. Such step detection permits to extract biologically meaningful information about the molecular system under study, such as the number of conformational states adopted and the corresponding interdye distances,58,59 the relative occurrence of conformations, as well as the rate coefficients at which they interconvert. In this study, we applied three HMMbased and two model-free step-detection algorithms to simulated FRET time traces. A number of experimentally relevant parameters were varied to assess their performance. A summary of the relative performances of the five algorithms is given in Figure 7A. In the first part, we applied each algorithm in conjunction with TDP fitting to assess the ability to infer the correct number and values of FRET states from simulated four-state trajectories, which were characterized by different SNRs. Our results demonstrate that STaSI in conjunction with ML-BIC fitting outperforms the other algorithms tested for SNR ≥ 4. The software faithfully reproduced the input model both in terms of the number of states and the corresponding FRETj values, with computation times that were comparably short. ebFRET achieved similar accuracies in restoring FRETj values, but an additional artefactual state was erroneously inferred. We believe that this degenerated state is due to the clock-induced charges introduced by the cameranoise model, but might also be induced by the Poisson-distributed fluorescence intensity data, both deviating from the assumed Gaussian PDF of the HMM-based approach. vbFRET and 25 ACS Paragon Plus Environment


HaMMy selected the appropriate model when the SNR of the trajectory was greater than 5. On the other hand, CPA in combination with ML-BIC selection turned out to be an inefficient approach. Overall performance follows the order STaSI > (ebFRET >) vbFRET > HaMMy > CPA (> ebFRET). In the second part, we studied the ability of each step-finding algorithm to correctly restore rate coefficients and its accuracy in detecting transitions. Here, both model-free approaches (STaSI and CPA) systematically underestimated transition rates, because a considerable number of fast transitions were missed in trajectories simulated under standard conditions (see Methods section for further details). In contrast, HaMMy, vbFRET and ebFRET make the assumption that state transitions are governed by exponential kinetics, and thus, were less prone to missing brief dwelltimes. In particular, ebFRET achieved the highest precision in determining rate coefficients and the highest accuracy in step detection for short trajectories. HaMMy outperformed the other algorithms in the case of unrealistically long trajectories and standard length, i.e. typical camera videos with 4000 frames, were best managed by ebFRET and vbFRET. At low SNR, vbFRET inferred both transition rates most precisely and displayed the highest accuracy in step detection. HaMMy significantly outperformed vbFRET and ebFRET, as it was able to resolve both forward and backward transition rates up to 0.1 frame–1, which correspond to an average state lifetime of ten time steps. It is important to stress that no algorithm evaluated here was able to resolve kinetics when one rate coefficient is greater than the frame rate. Two solutions exist to overcome this issue: (i) optimizing with care the experimental conditions in terms of laser power and exposure time60 and (ii) adding a Monte Carlo simulation step to recover time-averaged information.61 Finally, we showed that molecular heterogeneity in FRETj values and in total fluorescence intensity does not influence the determination of rate coefficients, most likely 26 ACS Paragon Plus Environment

Page 26 of 46

Page 27 of 46 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


because trajectories displaying lower FRET spacing (or very noisy states) are as likely to occur as trajectories with higher FRET spacing (or unrealistically well-defined states). However, an increase in sample heterogeneity caused a loss in step detection accuracy for all algorithms. This resulted in discarding some trajectories from rate coefficient calculations. Highest step detection accuracy and a minimal number of discarded trajectories was obtained by ebFRET, which is the only algorithm accounting for variations of FRETj values and transition rates across the sample among all algorithms tested. In conclusion, we propose the following strategy for statistical model selection in smFRET experiments (Figure 7B): (i) The number of FRET efficiency states Jopt should be determined with STaSI in combination with TDP fitting and ML-BIC selection. (ii) In order to accurately extract the rate coefficients associated with conformational changes, a HMM-based algorithm like HaMMy, ebFRET or vbFRET should be used and constrained to find Jopt states at maximum. It is important to mention that the results obtained from the evaluation strongly depends on the approach used to group states inferred from single trajectories. Here, we choose the BIC-limited TDP fitting for objective clustering, but other approaches like FRET histogram fitting nearest-neighbor grouping method

34

28

or the

have been successfully employed, and might lead to

differences in the outcome model. In this case, the present evaluation may serve as a template for further determination of optimal model selection strategies using different state grouping methods. 5. Supporting Information Supplementary information is available online. It contains a list of abbreviations and variables; additional information about state identification, which includes data formatting, model fitting and optimization, model selection procedure with ebFRET as well as the influence of camera27 ACS Paragon Plus Environment


noise on vbFRET performances; detailed information about the determination of transition rates, including rate constant calculation, the effect of the exclusion of outermost dwell-times and outof-range FRET values on algorithms performances. We provide three of the five algorithms tested in our MATLAB-based Multifunctional Analysis Software for Handling smFRET data (MASH-FRET). The simulation of intensity based FRET trajectories is part of the software. This software is freely available online under https://github.com/RNA-FRETools/MASH-FRET. 6.

Acknowledgements

Financial support by the European Research Council (ERC Starting Grant MIRNA N° 259092, to RKOS), the University of Zurich (to RKOS and Forschungskredit Grants FK-13-091, to MCASH; FK-14-096 and FK-15-095, to RB; FK-57010302, to SLBK; FK-13-108, to DK), as well as further financial support from the Swiss National Science Foundation, and the Swiss State Secretariat for Education and Research (COST Action CM1105, to RKOS) is gratefully acknowledged.


Page 28 of 46

Page 29 of 46 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


7. References (1) Ha, T.; Enderle, T.; Chemla, D. S.; Weiss, S. Dual-molecule spectroscopy: molecular rulers for the study of biological macromolecules. IEEE J. Sel. Top. Quantum Electron. 1996, 2, 1115– 1128, DOI: 10.1109/2944.577342. (2) Hohlbein, J.; Gryte, K.; Heilemann, M.; Kapanidis, A. N. Surfing on a new wave of singlemolecule fluorescence methods. Phys. Biol. 2010, 7, 031001, DOI: 10.1088/14783975/7/3/031001. (3) Börner, R.; Kowerko, D.; Miserachs, H. G.; Schaffer, M. F.; Sigel, R. K. Metal ion induced heterogeneity in RNA folding studied by smFRET. Coord. Chem. Rev. 2016, 327-328, 123–142, DOI: 10.1016/j.ccr.2016.06.002. (4) Tan, Y.-W.; Hanson, J. A.; Chu, J.-W.; Yang, H. Confocal single-molecule FRET for protein conformational dynamics 1084, 2014. (5) Börner, R.; Kowerko, D.; Hadzic, Mélodie C A S; König, Sebastian L B; Ritter, M.; Sigel, Roland K O. Simulations of camera-based single-molecule fluorescence experiments. PLoS One 2018, 13, e0195277, DOI: 10.1371/journal.pone.0195277. (6) Lee, N. K.; Kapanidis, A. N.; Wang, Y.; Michalet, X.; Mukhopadhyay, J.; Ebright, R. H.; Weiss, S. Accurate FRET measurements within single diffusing biomolecules using alternatinglaser excitation. Biophys. J. 2005, 88, 2939–2953, DOI: 10.1529/biophysj.104.054114. (7) Zhao, R.; Rueda, D. RNA folding dynamics by single-molecule fluorescence resonance energy transfer. Methods 2009, 49, 112–117, DOI: 10.1016/j.ymeth.2009.04.017. (8) Schuler, B.; Hofmann, H. Single-molecule spectroscopy of protein folding dynamics-expanding scope and timescales. Curr. Opin. Struct. Biol. 2013, 23, 36–47, DOI: 10.1016/j.sbi.2012.10.008. (9) Hwang, L. C.; Hohlbein, J.; Holden, S. J.; Kapanidis, A. N. Single-molecule FRET: methods and biological applications. In Handbook of single-molecule biophysics; Hinterdorfer, P., Oijen, A., Eds.; Springer US: New York, NY, 2009; pp 129–163. (10) Chung, H. S.; McHale, K.; Louis, J. M.; Eaton, W. A. Single-molecule fluorescence experiments determine protein folding transition path times. Science 2012, 335, 981–984, DOI: 10.1126/science.1215768. (11) Wang, Y.; Liu, Y.; Deberg, H. A.; Nomura, T.; Hoffman, M. T.; Rohde, P. R.; Schulten, K.; Martinac, B.; Selvin, P. R. Single molecule FRET reveals pore size and opening mechanism of a mechano-sensitive ion channel. eLife 2014, 3, e01834, DOI: 10.7554/eLife.01834. (12) Verbrugge, S.; Kapitein, L. C.; Peterman, Erwin J G. Kinesin moving through the spotlight: single-motor fluorescence microscopy with submillisecond time resolution. Biophys. J. 2007, 92, 2536–2545, DOI: 10.1529/biophysj.106.093575. (13) Hwang, H.; Myong, S. Protein induced fluorescence enhancement (PIFE) for probing protein-nucleic acid interactions. Chem. Soc. Rev. 2014, 43, 1221–1229, DOI: 10.1039/C3CS60201J. (14) Steffen, F. D.; Sigel, R. K. O.; Börner, R. An atomistic view on carbocyanine photophysics in the realm of RNA. Phys. Chem. Chem. Phys. 2016, 18, 29045–29055, DOI: 10.1039/C6CP04277E. 29 ACS Paragon Plus Environment


(15) Baltierra-Jasso, L. E.; Morten, M. J.; Laflör, L.; Quinn, S. D.; Magennis, S. W. Crowdinginduced hybridization of single DNA hairpins. J. Am. Chem. Soc. 2015, 137, 16020–16023, DOI: 10.1021/jacs.5b11829. (16) König, S. L. B.; Hadzic, M. C. A. S.; Fiorini, E.; Börner, R.; Kowerko, D.; Blanckenhorn, W. U.; Sigel, R. K. O. BOBA FRET: bootstrap-based analysis of single-molecule FRET data. PLoS One 2013, 8, e84157, DOI: 10.1371/journal.pone.0084157. (17) Schmid, S.; Götz, M.; Hugel, T. Single-molecule analysis beyond dwell times: demonstration and assessment in and out of equilibrium. Biophys. J. 2016, 111, 1375–1384, DOI: 10.1016/j.bpj.2016.08.023. (18) Lamichhane, R.; Solem, A.; Black, W.; Rueda, D. Single-molecule FRET of protein-nucleic acid and protein-protein complexes: surface passivation and immobilization. Methods 2010, 52, 192–200, DOI: 10.1016/j.ymeth.2010.06.010. (19) Kellner, R.; Hofmann, H.; Barducci, A.; Wunderlich, B.; Nettels, D.; Schuler, B. Singlemolecule spectroscopy reveals chaperone-mediated expansion of substrate protein. Proc. Natl. Acad. Sci. U. S. A. 2014, 111, 13355–13360, DOI: 10.1073/pnas.1407086111. (20) Borgia, A.; Kemplen, K. R.; Borgia, M. B.; Soranno, A.; Shammas, S.; Wunderlich, B.; Nettels, D.; Best, R. B.; Clarke, J.; Schuler, B. Transient misfolding dominates multidomain protein folding. Nat. Commun. 2015, 6, 8861, DOI: 10.1038/ncomms9861. (21) Sigel, R. K. O.; Sashital, D. G.; Abramovitz, D. L.; Palmer, A. G.; Butcher, S. E.; Pyle, A. M. Solution structure of domain 5 of a group II intron ribozyme reveals a new RNA motif. Nat. Struct. Mol. Biol. 2004, 11, 187–192, DOI: 10.1038/nsmb717. (22) Steiner, M.; Karunatilaka, K. S.; Sigel, R. K. O.; Rueda, D. Single-molecule studies of group II intron ribozymes. Proc. Natl. Acad. Sci. U. S. A. 2008, 105, 13853–13858, DOI: 10.1073/pnas.0804034105. (23) Steiner, M.; Rueda, D.; Sigel, R. K. O. Ca2+ induces the formation of two distinct subpopulations of group II intron molecules. Angew. Chem. Int. Ed. 2009, 48, 9739–9742, DOI: 10.1002/anie.200903809. (24) Adams, D. M.; Brus, L.; Chidsey, Christopher E. D.; Creager, S.; Creutz, C.; Kagan, C. R.; Kamat, P. V.; Lieberman, M.; Lindsay, S. et al. Charge Transfer on the Nanoscale: Current Status. J. Phys. Chem. B 2003, 107, 6668–6697, DOI: 10.1021/jp0268462. (25) Su, L. J.; Qin, P. Z.; Michels, W. J.; Pyle, A. M. Guiding ribozyme cleavage through motif recognition: the mechanism of cleavage site selection by a group ii intron ribozyme. J. Mol. Biol. 2001, 306, 655–668, DOI: 10.1006/jmbi.2000.4323. (26) Qin, F.; Auerbach, A.; Sachs, F. Hidden Markov modeling for single channel kinetics with filtering and correlated noise. Biophys. J. 2000, 79, 1928–1944, DOI: 10.1016/S00063495(00)76442-3. (27) McKinney, S. A.; Joo, C.; Ha, T. Analysis of single-molecule FRET trajectories using hidden Markov modeling. Biophys. J. 2006, 91, 1941–1951, DOI: 10.1529/biophysj.106.082487. (28) Bronson, J. E.; Fei, J.; Hofman, J. M.; Gonzalez, R. L.; Wiggins, C. H. Learning rates and states from biophysical time series: a Bayesian approach to model selection and single-molecule FRET data. Biophys. J. 2009, 97, 3196–3205, DOI: 10.1016/j.bpj.2009.09.031. 30 ACS Paragon Plus Environment

Page 30 of 46

Page 31 of 46 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


(29) Greenfeld, M.; Pavlichin, D. S.; Mabuchi, H.; Herschlag, D. Single Molecule Analysis Research Tool (SMART): an integrated approach for analyzing single molecule data. PLoS One 2012, 7, e30024, DOI: 10.1371/journal.pone.0030024. (30) van de Meent, Jan-Willem; Bronson, J. E.; Wiggins, C. H.; Gonzalez, R. L. Empirical Bayes methods enable advanced population-level analyses of single-molecule FRET experiments. Biophys. J. 2014, 106, 1327–1337, DOI: 10.1016/j.bpj.2013.12.055. (31) Preus, S.; Noer, S. L.; Hildebrandt, L. L.; Gudnason, D.; Birkedal, V. iSMS: singlemolecule FRET microscopy software. Nat. Methods 2015, 12, 593–594, DOI: 10.1038/nmeth.3435. (32) Juette, M. F.; Terry, D. S.; Wasserman, M. R.; Altman, R. B.; Zhou, Z.; Zhao, H.; Blanchard, S. C. Single-molecule imaging of non-equilibrium molecular ensembles on the millisecond timescale. Nat. Methods 2016, 13, 341–344, DOI: 10.1038/nmeth.3769. (33) van de Meent, Jan-Willem; Bronson, J. E.; Wood, F.; Gonzalez, R. L.; Wiggins, C. H. Hierarchically-coupled hidden Markov models for learning kinetic rates from single-molecule data. JMLR Workshop Conf. Proc. 2013, 28, 361–369. (34) Shuang, B.; Cooper, D.; Taylor, J. N.; Kisley, L.; Chen, J.; Wang, W.; Li, C. B.; Komatsuzaki, T.; Landes, C. F. Fast step transition and state identification (STaSI) for discrete single-molecule data analysis. J. Phys. Chem. Lett. 2014, 5, 3157–3161, DOI: 10.1021/jz501435p. (35) Hua, W.; Young, E. C.; Fleming, M. L.; Gelles, J. Coupling of kinesin steps to ATP hydrolysis. Nature 1997, 388, 390–393, DOI: 10.1038/41118. (36) Kerssemakers, Jacob W J; Munteanu, E. L.; Laan, L.; Noetzel, T. L.; Janson, M. E.; Dogterom, M. Assembly dynamics of microtubules at molecular resolution. Nature 2006, 442, 709–712, DOI: 10.1038/nature04928. (37) Sadler, B. M.; Swami, A. Analysis of multiscale products for step detection and estimation. IEEE Trans. Inform. Theory 1999, 45, 1043–1051, DOI: 10.1109/18.761341. (38) Carter, B. C.; Vershinin, M.; Gross, S. P. A comparison of step-detection methods: how well can you do? Biophys. J. 2008, 94, 306–319, DOI: 10.1529/biophysj.107.110601. (39) Pirchi, M.; Ziv, G.; Riven, I.; Cohen, S. S.; Zohar, N.; Barak, Y.; Haran, G. Single-molecule fluorescence spectroscopy maps the folding landscape of a large protein. Nat. Commun. 2011, 2, 493, DOI: 10.1038/ncomms1504. (40) Zhuang, X.; Kim, H.; Pereira, Miguel J B; Babcock, H. P.; Walter, N. G.; Chu, S. Correlating structural dynamics and function in single ribozyme molecules. Science 2002, 296, 1473–1476, DOI: 10.1126/science.1069013. (41) Lee, J. Y.; Okumus, B.; Kim, D. S.; Ha, T. Extreme conformational diversity in human telomeric DNA. Proc. Natl. Acad. Sci. U. S. A. 2005, 102, 18938–18943, DOI: 10.1073/pnas.0506144102. (42) Fiorini, E.; Börner, R.; Sigel, R. K. O. Mimicking the in vivo Environment – The effect of crowding on RNA and biomacromolecular folding and activity. CHIMIA 2015, 69, 207–212, DOI: 10.2533/chimia.2015.207.



(43) Holden, S. J.; Uphoff, S.; Hohlbein, J.; Yadin, D.; Le Reste, L.; Britton, O. J.; Kapanidis, A. N. Defining the limits of single-molecule FRET resolution in TIRF microscopy. Biophys. J. 2010, 99, 3102–3111, DOI: 10.1016/j.bpj.2010.09.005. (44) Kowerko, D.; König, S. L. B.; Skilandat, M.; Kruschel, D.; Hadzic, M. C. A. S.; Cardo, L.; Sigel, R. K. O. Cation-induced kinetic heterogeneity of the intron-exon recognition in single group II introns. Proc. Natl. Acad. Sci. U. S. A. 2015, 112, 3403–3408, DOI: 10.1073/pnas.1322759112. (45) Taylor, J. N.; Landes, C. F. Improved resolution of complex single-molecule FRET systems via wavelet shrinkage. J. Phys. Chem. B 2011, 115, 1105–1114, DOI: 10.1021/jp1050707. (46) Ha, T.; Tinnefeld, P. Photophysics of fluorescent probes for single-molecule biophysics and super-resolution imaging. Annu. Rev. Phys. Chem. 2012, 63, 595–617, DOI: 10.1146/annurevphyschem-032210-103340. (47) König, S. L. B.; Liyanage, P. S.; Sigel, R. K. O.; Rueda, D. Helicase-mediated changes in RNA structure at the single-molecule level. RNA Biol. 2013, 10, 133–148, DOI: 10.4161/rna.23507. (48) Lerner, E.; Ploetz, E.; Hohlbein, J.; Cordes, T.; Weiss, S. A Quantitative theoretical framework for protein-induced fluorescence enhancement-Förster-type resonance energy transfer (PIFE-FRET). J. Phys. Chem. B. 2016, 120, 6401–6410, DOI: 10.1021/acs.jpcb.6b03692. (49) Kelly, D.; Dillingham, M.; Hudson, A.; Wiesner, K. A new method for inferring hidden markov models from noisy time sequences. PLoS One 2012, 7, e29703, DOI: 10.1371/journal.pone.0029703. (50) Bronson, J. E.; Hofman, J. M.; Fei, J.; Gonzalez, R. L.; Wiggins, C. H. Graphical models for inferring single molecule dynamics. BMC Bioinf. 2010, 11, S2, DOI: 10.1186/1471-2105-11S8-S2. (51) Taylor, Wayne A. Change-point analysis: a powerful new tool for detecting changes; online, 2017. (52) Efron, B. Bootstrap methods: another look at the jackknife. Ann. Statist. 1979, 7, 1–26, DOI: 10.1214/aos/1176344552. (53) MacQueen, J. Some methods for classification and analysis of multivariate observations. In Statistics; Le Cam, L. M., Neyman, J., Eds., 1967; pp 281–297. (54) Schmied, J. J.; Gietl, A.; Holzmeister, P.; Forthmann, C.; Steinhauer, C.; Dammeyer, T.; Tinnefeld, P. Fluorescence and super-resolution standards based on DNA origami. Nat. Methods 2012, 9, 1133–1134, DOI: 10.1038/nmeth.2254. (55) Dimura, M.; Peulen, T. O.; Hanke, C. A.; Prakash, A.; Gohlke, H.; Seidel, C. A. Quantitative FRET studies and integrative modeling unravel the structure and dynamics of biomolecular systems. Curr. Opin. Struct. Biol. 2016, 40, 163–185, DOI: 10.1016/j.sbi.2016.11.012. (56) Hirsch, M.; Wareham, R. J.; Martin-Fernandez, M. L.; Hobson, M. P.; Rolfe, D. J. A stochastic model for electron multiplication charge-coupled devices‐from theory to practice. PLoS One 2013, 8, e53671, DOI: 10.1371/journal.pone.0053671.


Page 32 of 46

Page 33 of 46 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


(57) Liu, Y.; Park, J.; Dahmen, K. A.; Chemla, Y. R.; Ha, T. A comparative study of multivariate and univariate hidden Markov modelings in time-binned single-molecule FRET data analysis. J. Phys. Chem. B 2010, 114, 5386–5403, DOI: 10.1021/jp9057669. (58) Stein, I. H.; Steinhauer, C.; Tinnefeld, P. Single-molecule four-color FRET visualizes energy-transfer paths on DNA origami. J. Am. Chem. Soc. 2011, 133, 4193–4195, DOI: 10.1021/ja1105464. (59) Schuler, B.; Lipman, E. A.; Steinbach, P. J.; Kumke, M.; Eaton, W. A. Polyproline and the "spectroscopic ruler" revisited with single-molecule fluorescence. Proc. Natl. Acad. Sci. U. S. A. 2005, 102, 2754–2759, DOI: 10.1073/pnas.0408164102. (60) Zhuang, X.; Bartley, L. E.; Babcock, H. P.; Russell, R.; Ha, T.; Herschlag, D.; Chu, S. A Single-molecule study of RNA catalysis and folding. Science 2000, 288, 2048–2051, DOI: 10.1126/science.288.5473.2048. (61) Chen, J.; Pyle, J. R.; Sy Piecco, Kurt Waldo; Kolomeisky, A. B.; Landes, C. F. A two-step method for smFRET data analysis. J. Phys. Chem. B. 2016, 120, 7128–7132, DOI: 10.1021/acs.jpcb.6b05697.



8. Figures

Figure 1. Assessment of different algorithms in terms of their ability to correctly infer the number of states and the corresponding mean FRET values (model determination), as well as to faithfully detect state transitions (transition detection). A) Workflow underlying model determination: (i) Representative four-state FRET trajectory J0 = 4 derived from simulated data (light gray). The 34 ACS Paragon Plus Environment

Page 34 of 46

Page 35 of 46 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


trajectory was analyzed with a step-finding algorithm, yielding an inferred state trajectory (solid black line). Dotted lines indicate state transitions in the simulated trajectory. Each data set comprised 100 trajectories yielding 100 inferred state trajectories. (ii) Inferred state trajectories were randomized via bootstrapping, yielding 100 bootstrap samples per data set. (iii) Each bootstrap sample was used to build a TDP, i.e., a contour plot in which FRETbefore is plotted against FRETafter. Each TDP was Gaussian-convoluted and subjected to model selection. (iv) The selected model (Jm = 4, mixture of 16 Gaussians) is indicated in gray (black dots indicate Gaussian centers). (v) The mean number of states J̄ and the corresponding standard deviation σJ were computed from all 100 inferred models. B) Workflow underlying transition detection: (i) Representative two-state FRET trajectory J0 = 2 derived from simulated data (light gray). Dotted lines indicate state transitions in the simulated trajectory. The trajectory was analyzed with a stepfinding algorithm, yielding an inferred state trajectory (solid black line). Dwell times were classified as “open” or “closed” as illustrated in the case of a dwell time ∆topen in the open state and ∆tclosed in the closed state. (ii) A total of 100 inferred trajectories were generated for each data set, followed by bootstrapping, yielding 100 bootstrap samples per data set. For each bootstrap sample, (iii) dwell-time histograms of ∆topen and ∆tclosed were generated (black bars) and fitted to monoexponential decay functions (iv) yielding transition rates kopening and kclosing. Here, 1/kopening and 1/ kclosing correspond to the time that is required for the exponential decay to decrease to 1/e of its amplitude at ∆t = 0. (v) Mean rate coefficients k̅,i and standard deviations σk,i associated with opening and closing were calculated from all 100 inferred histogram fits.16 C) Inferred models, thus number of states and corresponding FRET value, were compared to the ground truth (GT, black boxes) and classified as true positive (TP, blue), false positive (FP, red) and false negative (FN, orange). D) Detected transitions (gray line) were compared to GT (black dashed



line) and classified as TPs (blue), FPs (red) or FNs (orange). Please refer to the methods section for further details. Default simulation parameters: L = 4200 frames; SNR = 3.5; (i) model determination: J0 = 4, FRETj,,0 = 0.3, 0.45, 0.6, and 0.8, and k12 = k21 = 0.05 s–1, k23 = k32 = 0.13 s– 1

, k34 = k43 = 0.03 s–1; (ii) transition detection: J0 = 2, FRETj,,0 = 0.3 (open) and 0.7 (closed), and

kopening = 0.02 frame–1 and kclosing,0 = 0.05 frame–1; wFRET = 0 and wtot,0 = 0.


Page 36 of 46

Page 37 of 46 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Figure 2. Evaluation of step-finding algorithms in terms of their ability to restore the correct number of FRET efficiency states at different SNR =1, … 7. A) Mean number of inferred states J̄ with error bars set by σJ at different SNRs. The dashed line indicates the ground truth. Model-free approaches (upper panel) and HMM-based approaches (lower panel) are presented. B) Mean accuracy, precision and recall of inferred FRET values at different SNRs. Crosses indicated nonconvergence of the model or software errors. C) Computation time tcomp as a function of the true 37 ACS Paragon Plus Environment


number of states J0. Colored swathed indicate the mean value within the range set by the standard deviation calculated from 100 trajectories. Default simulation parameters: L = 4200 frames; J0 = 4, SNR = 3.5; FRETj,,0 = 0.3, 0.45, 0.6, and 0.8, and k12 = k21 = 0.05 s–1, k23 = k32 = 0.13 s–1, k34 = k43 = 0.03 s–1; wFRET = 0 and wtot,0 = 0.


Page 38 of 46

Page 39 of 46 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Figure 3. Representative simulated FRET trajectories (gray) and the corresponding state trajectories (black) as determined using different step finding algorithms. A) CPA at SNR = 7. Red arrows highlight erroneously identified blur states in the state trajectory. B) and C) vbFRET at SNR = 3. Red arrows indicate spurious transitions to artefactual FRET states leading for B) to probability clusters centered around the FRETafter = FRETbefore diagonal in TDPs and for C) to clusters at extreme FRET values for vbFRET probably due to clock-induced charge (CIC). D) HaMMy at SNR = 3. Excursions to one of three virtual states (dotted red lines) are highlighted with red arrows. In all graphs, ground truth FRETj,0. values are represented as horizontal gray lines. Default simulation parameters: L = 4200 frames; J0 = 4; FRETj,,0 = 0.3, 0.45, 0.6, and 0.8, and k12 = k21 = 0.05 s–1, k23 = k32 = 0.13 s–1, k34 = k43 = 0.03 s–1; wFRET = 0 and wtot,0 = 0. 39 ACS Paragon Plus Environment



Page 40 of 46

Page 41 of 46 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Figure 4. Evaluation of step finding algorithms in terms of their ability to restore rate coefficients and to correctly identify transitions. (Top) Examples for varied simulation parameters from the left to the right: the trace length L = 70, …, 64000 frames, kclosing,0 = 0.02, …, 20 frame–1, SNR = 1, …, 7, the FRET heterogeneity width wFRET = 0 … 0.15, and the total fluorescence intensity heterogeneity width wtot,0 = 0, …, 50%. A) Deviations for the slow rate coefficient ∆kopening and B) for the fast rate coefficient ∆kclosing. Colored swaths denote the relative deviation of inferred and true rate coefficient ±3σk,i. Model-free approaches (upper panels), HMM-based approaches (lower panels). Missing data are due to an insufficient number of transition detections, and thus dwell-times, to fit an exponential function to the dwell-time histogram. C) Mean step detection accuracy, precision and recall averaged over all trajectories generated for one input parameter set. Default simulation parameters: L = 4200 frames; J0 = 2, FRETj,,0 = 0.3 (open) and 0.7 (closed), and kopening = 0.02 frame–1 and kclosing,0 = 0.05 frame–1; SNR = 3.5; wFRET = 0 and wtot,0 = 0.



Figure 5. Quantifying the influence of heterogeneous FRET broadening on the performance of step finding algorithms. A) Input model and simulation output at wFRET = 0.15, which characterizes unrealistically high FRET value heterogeneity. The two Gaussian probability density functions centered at FRET = [0.3, 0.7] and depicted as black dotted lines. Mean FRET values of a set of 100 FRET trajectories are shown in gray. Letters "a" and "b" highlight the mean FRET values of the representative trajectories shown in B) yielding small "a" and large "b" differences in FRET values due to heterogeneous FRET broadening. C) Number of trajectories considered by CPA (black), STaSI ( red), ebFRET (cyan), vbFRET (green), and HaMMy (blue) for the calculation of rate coefficients. Static traces devoid of transitions were excluded. D) State trajectory of trace "a" shown in panel B as determined with CPA (black) and STaSI ( red) as model free (top) and ebFRET (cyan), vbFRET (green), and HaMMy (blue) as model-based (bottom) step finding algorithms. The ground truth is depicted as a dashed black line. Default simulation parameters: L = 4200 frames; SNR = 3.5; J0 = 2, FRETj,,0 = 0.3 (open) and 0.7 (closed), and kopening = 0.02 frame–1 and kclosing,0 = 0.05 frame–1; wtot,0 = 0.


Page 42 of 46

Page 43 of 46 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Figure 6. Quantifying the influence of heterogeneous fluorescence intensity broadening on the performance of step finding algorithms. A) Input model and simulation output at wtot,0 = 17.3 pc/frame, which characterizes unrealistically high fluorescence intensity heterogeneity. The Gaussian probability density function is centered at Itot,0 = 34.6 pc/frame and shown as a black dotted line. Mean total intensities of a set of 100 FRET trajectories are given in gray. Letters indicate the representative trajectories "a" and "b" shown in panel B). C) Number of trajectories considered by ebFRET (cyan), vbFRET (green), HaMMy (blue), STaSI (red) and CPA (black) for the calculation of rate coefficients. Static traces devoid of transitions were excluded. D) State trajectory of trace "a" shown in panel B as determined with CPA (black) and STaSI ( red) as model free (top) and ebFRET (cyan), vbFRET (green), and HaMMy (blue) as model-based (bottom) step finding algorithms. The ground truth is depicted as a dashed black line. Default simulation parameters: L = 4200 frames; SNR = 3.5; J0 = 2, FRETj,,0 = 0.3 (open) and 0.7 (closed), and kopening = 0.02 frame–1 and kclosing,0 = 0.05 frame–1; wFRET = 0.



Figure 7. Summary of algorithm performances. A) score ranging from 0 to 5 points and rating the goodness of the outcome in performing different tasks was attributed to each algorithm. Red: 44 ACS Paragon Plus Environment

Page 44 of 46

Page 45 of 46 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


state identification. Black: computation time. Remaining colors are used for step detection upon variation of specific simulation parameters (orange: trace length, yellow: signal-to-noise ratio, green: rate coefficient, blue: heterogeneous FRET broadening, violet: heterogeneity in SNR). A score of 0 point is attributed to a failure (e.g. CPA and ebFRET always estimate the wrong number of states) and 5 points to the best-performing algorithm (e.g. STaSI estimates the number of states best). B) Flow chart for model selection and transition detection. Depending on the particular performance in terms of accuracy and precision of the five algorithms tested, the model determination (1st step, accuracy > 0.5 and precision > 0.9) and transition detection (2nd step, accuracy > 0.8, precision > 0.8 and ∆k < 25%) should be performed according to the individual experimental parameters, such as SNR etc. CPA and ebFRET should not be considered for model selection; CPA (and STaSI) should not be considered for transition detection.



9. TOC Graphic


Page 46 of 46

Research article Reliable State Identification and State Transition

Recommend Documents