Good versus Good Enough? Empirical tests of methane leak detection

Subscriber access provided by READING UNIV

Article

Good versus Good Enough? Empirical tests of methane leak detection sensitivity of a commercial infrared camera Arvind P. Ravikumar, Jingfan Wang, Mike McGuire, Clay S. Bell, Daniel Zimmerle, and Adam R. Brandt Environ. Sci. Technol., Just Accepted Manuscript • DOI: 10.1021/acs.est.7b04945 • Publication Date (Web): 19 Jan 2018 Downloaded from http://pubs.acs.org on January 20, 2018

Just Accepted “Just Accepted” manuscripts have been peer-reviewed and accepted for publication. They are posted online prior to technical editing, formatting for publication and author proofing. The American Chemical Society provides “Just Accepted” as a free service to the research community to expedite the dissemination of scientific material as soon as possible after acceptance. “Just Accepted” manuscripts appear in full in PDF format accompanied by an HTML abstract. “Just Accepted” manuscripts have been fully peer reviewed, but should not be considered the official version of record. They are accessible to all readers and citable by the Digital Object Identifier (DOI®). “Just Accepted” is an optional service offered to authors. Therefore, the “Just Accepted” Web site may not include all articles that will be published in the journal. After a manuscript is technically edited and formatted, it will be removed from the “Just Accepted” Web site and published as an ASAP article. Note that technical editing may introduce minor changes to the manuscript text and/or graphics which could affect content, and all legal disclaimers and ethical guidelines that apply to the journal pertain. ACS cannot be held responsible for errors or consequences arising from the use of information contained in these “Just Accepted” manuscripts.

Environmental Science & Technology is published by the American Chemical Society. 1155 Sixteenth Street N.W., Washington, DC 20036 Published by American Chemical Society. Copyright © American Chemical Society. However, no copyright claim is made to original U.S. Government works, or works produced by employees of any Commonwealth realm Crown government in the course of their duties.

Page 1 of 18

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38

Environmental Science & Technology

“Good versus Good Enough?” Empirical tests of methane leak detection sensitivity of a commercial infrared camera Arvind P. Ravikumar*,†, Jingfan Wang†, Mike McGuire‡, Clay S. Bell‡, Daniel Zimmerle§, Adam R. Brandt† †Department

of Energy Resources Engineering, Stanford University, 367 Panama St., Stanford,

CA 94305 ‡Colorado State University Energy Institute, 430 North College Av., Fort Collins, CO 80542 §Department of Mechanical Engineering, Colorado State University, 1374 Campus Delivery, Fort Collins, CO 80523 *Corresponding author: Phone: (650) 736-3491, Email address: [email protected]

Abstract Methane – a key component of natural gas – is a potent greenhouse gas. A key feature of recent methane mitigation policies is the use of periodic leak detection surveys, typically done with optical gas imaging (OGI) technologies. The most common OGI technology is an infrared camera. In this work, we experimentally develop detection probability curves for OGI-based methane leak detection under different environmental and imaging conditions. Controlled single blind leak detection tests show that the median detection limit (50% detection likelihood) for FLIR-camera based OGI technology is about 20 g CH4/h at an imaging distance of 6 m, an order of magnitude higher than previously reported estimates of 1.4 g CH4/h. Furthermore, we show that median and 90% detection likelihood limit follows a power-law relationship with imaging distance. Finally, we demonstrate that real-world marginal effectiveness of methane mitigation through periodic surveys approaches zero as leak detection sensitivity improves. For example, a median detection limit of 100 g CH4/h is sufficient to detect the maximum amount of leakage that is possible through periodic surveys. Policy makers should take note of these limits while designing equivalence metrics for next-generation leak detection technologies that can trade sensitivity for cost without affecting mitigation priorities.

ACS Paragon Plus Environment


39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76

Introduction The resurgent role of natural gas in the US energy sector has invited debate over the most effective path towards a low-carbon future. With recent advances in horizontal drilling and hydraulic fracturing, the use of natural gas, especially in the electricity sector, is rapidly expanding [1]. Fugitive methane emissions and leaks across the natural gas supply chain can chip away at the greenhouse gas (GHG) emissions advantage of gas over coal [2]. Technology lock-in, for example, in the space- and water-heating sectors, further underscores the likelihood of continued gas use in the future. It is therefore important to mitigate methane emissions from this sector until more sustainable and cost-effective options can take over. Recent work has greatly improved our understanding of methane emissions from the natural gas sector. Experimental studies of leakage have been performed at production sites [3, 4, 5] gathering and processing plants [6], transportation and storage compressor stations [7], distribution lines [8, 9], and power plants [10]. These studies find a few common results. First, emissions are broadly distributed across the supply chain, and no single sector/source dominates [11] (see for example the most recent greenhouse gas inventory (GHGI) from the US Environmental Protection Agency (EPA) [12]). Second, intermittent or infrequent sources can contribute to emissions but are difficult to sample in conventional leak detection surveys [13]. Third, leak-size distributions are highly skewed – a small number of high-emitting sources, often called ‘super-emitters’, account for a large fraction of emissions [14]. These studies have informed U.S. federal regulations aimed at mitigating methane emissions from the oil and gas sector [15]. In 2016, the U.S. EPA updated the 2012 New Source Performance Standards [16] to require periodic leak detection and repair (LDAR) programs. These surveys must be performed semi-annually at production well pads, and quarterly at larger sites like mid-stream compressor stations. EPA recommends the use of optical gas imaging (OGI) techniques for these surveys based on long-term experience of operators with the technology and due to positive feedback from operators in Colorado [17]. OGI makes hydrocarbon plumes visible to operators, allowing contractors to diagnose faulty components [18]. EPA estimates that the use of OGIbased LDAR would result in fugitive emissions reductions of 60% for well-pads and 80% for midstream compressor stations, compared to baseline emissions [19]. We showed recently that these estimated emissions reductions may be overly optimistic due to uncertainties in technology performance and emissions characteristics [20]. Many operators have adopted OGI for their LDAR survey due to its convenience and lack of EPAapproved cost-effective alternative technologies. However, aside from anecdotal evidence [17], there has been little systematic empirical study of the limits and effectiveness of OGI in realworld conditions. Benson et al. performed a series of lab experiments under carefully controlled


Page 2 of 18

Page 3 of 18

77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114


laboratory conditions to determine minimum detectable leak rates (MDLR) for a variety of hydrocarbons [21]. They found detection limits of 0.8 g/h for pure methane, and 3.4 g/h in a nitrogen mixture, at a standoff distance of 3 m. EPA, on the other hand, prescribes a minimum detection limit of 30 g/h for a 50-50 methane-propane mixture as being equivalent to OGI performance [16]. This study aims to improve our understanding of OGI technologies through independent empirical analysis. We examine metered leaks from a controlled-release facility that replicates a natural gas production well-site. Leaks are examined under different environmental and imaging conditions by a trained OGI-operator. Leak-sizes are not disclosed to the operator during the experiments, and are independently controlled in a randomized single-blind study design. We use the results to generate functions for detection probability as a function of leak size and distance. We also examine the role of wind conditions and other environmental factors. We conclude with implications for mitigation policy. Methods We performed experiments at the Methane Emissions Technology Evaluation Center (METEC) at Colorado State University in Fort Collins CO over five days July 10-14, 2017. METEC is a controlled release facility designed to test US Department of Energy funded methane detection technologies through the MONITOR program. METEC is also open to other users. The site contains equipment typically found at a natural gas production site including wellheads, separators, and tank batteries. Each piece of equipment has multiple leak points made from ¼” steel tubing. Methane is sourced from a centrally-located cylinder at 2500 psi and flow is controlled by a combination of pressure regulation plus 3 parallel discrete choked-flow orifices that in combination allow for 8 flow rates from each source. The pressure-dependent flow rate is calibrated for each leak source across a range of input pressures (see S2 in SI). Flow rates varied from 33 – 330 g /h (2 – 20 standard cubic feet per hour). Participating researcher (JW) underwent methane leak-detection training and OGI certification at Infrared Training Center (Dallas, TX). This training emphasized fundamentals of OGI physics, practical detection guidance, and best practices for imaging methane leaks at oil and gas facilities. We used a FLIR GF-320 GasFind IR camera for leak detection [22]. Throughout this work, JW, henceforth referred to as the operator, detected leaks following training protocols, while AR and MM controlled the leak-size. According to EPA, OGI based leak detection surveys require a daily calibration using a known methane emission rate at a distance not exceeding the survey distance [16]. We did not perform this calibration because the objective was to identify leak detection probability at different imaging distances, not to ensure detection certainty.



115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152

One-minute average weather data were recorded during experiments. These include wind speed and direction (using a 3-axis sonic anemometer), temperature, humidity, and solar irradiance. Weather recordings were made in a single location, 20 - 100 m away from the leaks being imaged. In addition, 5-minute average weather information was also collected from the nearby Christman Field weather station, located 100 m to the south of the field site. Conditions were generally warm throughout the week (30 – 35 ⁰C) with mild winds (1 – 4 m/s), and backgrounds ranging from clear skies to fully overcast (see SI). The measurement period was longer than any shortterm wind gusts and did not adversely affect the results. Methane was released from three locations at the METEC site: a wellhead (flange), separator (Kimray vent), and wellhead (valve) – see SI for images. The leaks had both diffuse (flange) and point-source characteristics (vent and valve). In this work, the operator knew the leak locations a priori, a practice unlike typical leak detection surveys. This was chosen to isolate the effect of the skill level of the operator in finding leaks from the visual detection limits of the camera. Future studies can now use the data presented here to understand the effects of the operator on leak detection. To test the entire detection probability curve, we performed experiments under two different categories: small leaks at short distances (SSD), and large leaks at longer distances (LLD). The SSD experiments examined leaks in the 3 – 83 g/h (0.2 – 5 scfh) range. The LLD experiments ranged from 17 – 332 g/h (1 to 20 scfh). See Table [1]. For both SSD and LLD experiments, a “set” of eight unique leaks were randomly released at each equipment, including a null leak rate (0 g/h). 10 such sets were performed at each leak location (3) and each imaging distance (3 distances for SSD, 4 distances for LLD, see Table [1]). Combinations of results give detection probabilities for different distances and sizes. Each leak is released for approximately 2 minutes, during which the operator stands at various distances and notes with a binary measure [0-1, not-visible/visible] whether the leak is visible through the camera. Because leaks created by pressure regulated orifices or a mass flow controller may have different visualization characteristics than leaks form real equipment, only binary results were allowed from the operator (i.e., no “possible plume” results were allowed). Given the discretization into leak size regimes (2), leak sizes (8), imaging distances (7), and repeat sets (10), each piece of equipment is imaged 560 times, for a total of 1680 binary observations across all experiments. Short video clips (10 – 15 s) were made of all binary “yes” observations for later analysis (see section S3 in SI). The operator checks for leaks in the ‘high sensitivity’ mode of the OGI camera following standard industry practices. The ‘high-sensitivity’ mode uses frame differencing to emphasize motion. Surveying four distances generally takes approximately 2 minutes (≈2 components per minute). This is a conservative detection speed, as one prior work suggests practical OGI survey rates of ≈8 components per minute [23]. However, there is


Page 4 of 18

Page 5 of 18

153 154 155 156 157 158 159 160

161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183


significant variation in actual industry practices as measurement speed depends on prior knowledge (e.g., whether a component was found leaking in previous surveys leading to careful future observations), gas field operator and their mitigation priorities, size of the facility, and cost of the program. Table 1. Characteristics of SSD and LLD leak settings. The leak-rate error in the SSD and LLD categories correspond to accuracy of the mass-flow controller (~ 2 g/h), and one standard deviation of measured leak rate data recorded at 1 Hz, respectively. Leak Category Approx. leak sizes Approx. Leak Sizes Distances Distances [ft] [g/h] [scfh] [m] Small-short0, 1.7, 4.1, 8.3, 0, 0.1, 0.25, 0.5, 1.5, 3, 6 5, 10, 20 distance (SSD) 12.4, 17, 33, 83 0.75, 1, 2, 5 Large-long10, 40, 95, 126, 0.6, 2.4, 5.7, 7.6, 6, 9, 12, 15 20, 30, 40, 50 distance (LLD) 163, 196, 260, 295 9.8, 11.8, 15.7, 17.8 Results Measurements at each imaging distance are combined across the three leak locations to give detection probability values for each leak size. Measured leak volumes in scfh are converted to gCH4/h (24.465 L/mol at 25 ⁰C, 86.7 average mol% CH4). Figure 1 shows empirical detection probability curves at different imaging distances for the SSD (Figure 1(a)) and LLD (Figure 1(b)) experiment. The horizontal error bars show uncertainty in methane flow rates. For the SSD measurements, this corresponds to the accuracy rating of the mass flow controller (standard deviation of 1% of full scale reading). For the LLD measurements, error rates correspond to 1 standard deviation of the mean leak rate data that were automatically recorded at 1 Hz (See SI section S2). The vertical error bars correspond to uncertainty associated with finite sample sizes (n ≈ 30), calculated assuming a binomial distribution. It corresponds to uncertainty at the 95% confidence interval level (±2σ). In figure 1(a), smaller leak rates (< ≈10 g/h) have higher error rates because of the precision of the mass flow controller used in the experiment. The median detection limit, defined as the leak rate at which the probability of detection is 50%, is 3, 6, 20 g/h, respectively for imaging distances of 1.5, 3, and 6 m. Similarly, figure 1(b) shows the detection probability curves for longer imaging distances (6 m through 15 m). Here, the median detection limits are 18, 51, 129, and 151 g/h, for imaging distances of 6, 9, 12, and 15 m, respectively. Data taken under various weather conditions (see SI section S3) were aggregated in deriving these plots.



184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199

Wind speed is a key parameter that can affect leak detection. During the course of this study, 90% of 5-minute average wind-speed were less than 4.3 m/s, with an experiment-averaged mean wind-speed of 3 ± 1.3 m/s (1). Moderate wind speeds have been observed during leak detection surveys at natural gas facilities. The Fort Worth Air Quality Study in Texas [3], with reported wind speeds during ~2000 individual leak rate measurements, observed an average wind speed of 2.3 m/s (see SI section S4). Component-level measurements in bottom-up studies (and in our study) happen in an environment of equipment ‘clutter’ that affects wind flow. Therefore, actual wind speeds near the leak tend to be lower than wind speeds recorded by weather stations. High wind speeds are likely to be relevant when measuring equipment that are taller (e.g., tanks) where winds are not as obstructed – our work did not include detection limits for leaks from tanks. In this study, we did not find any significant differences in leak detection probability as a function of wind-speed (see SI section S4). By corollary, wind speeds below 4.3 m/s (≈ 10 mph) should likely be acceptable for leak detection. Future experiments should help identify the relationship between high wind speeds (> 5 m/s) and leak detection probability.

200


Page 6 of 18

Page 7 of 18

201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226


Figure 1. Experimentally-derived leak detection probability curves at various imaging distances. (a) Detection probability curves for the small-short-distance (SSD) experiments. Median detection limits are (µ±1σ) 3±0.8, 6±0.8, 20±0.8 g/h for imaging distances of 1.5, 3, and 6 m, respectively. (b) Detection probability curves for the long-large-distance (LLD) experiments. The median detection limits are approx. 18±2, 51±5, 129±10, and 151±8 g/h at imaging distances 6, 9, 12, and 15 m, respectively. The horizontal error bars in (a) corresponds to the accuracy of the mass flow controller, while the error bars in (b) correspond to 1 standard deviation of the mean in observed flow rates. The vertical error bars correspond to 95% confidence interval associated with the finite sample size (µ±2σ). The mean size of each leak bin in the experiment, transformed to logarithmic magnitudes (log10), is used as a size measure. The empirical probability of detection is then fitted with a sigmoidal Gaussian cumulative probability function: 𝑓=

1 1 + exp⁡[−𝑘(𝑥 − 𝑥0 )]

where f is the fraction of leaks detected (0 – 1), x is the log-transformed leak magnitude [log10(g CH4/h)], and k and x0 are fitting constants. k, in units of 1/[log10(g CH4/h)], effectively denotes the ‘steepness’ of the detection probability curve while x0 [log10(g CH4/h)] is the median detection limit where f = 0.5. To fit the probability function f, we incorporated two aspects of uncertainty – uncertainty arising from experimental errors (both leak-sizes and probability estimates as shown in Figure 1), and uncertainty arising from fitting the data to sigmoid function. First, we incorporate uncertainty in



227 228 229 230 231 232 233 234 235

236 237 238 239 240 241

leak volume and detection probability shown in Figure 1. To do this, we sample the observed detection probability for 1000 realizations for each leak volume, incorporating noise around each experimentally observed data-point in Figure 1. For each realization, we solve for the best fitting values of k and x0. Secondly, we incorporate curve fitting noise by drawing each plotted k and x0 from distributions created from best-fitting k and x0 and respective standard errors (SE). The drawn values for k and x0 are then used to generate 1000 independent cumulative probability curves for each distance. See Figure 2 for fitting results. See Table [2] for values of k, x0 and SEs for the fit to median leak volume.

Figure 2: Sigmoidal cumulative detection probabilities as a function of log-transformed leak size. Fitted curves as lines, measurements as dots. See Table 2 for best fitting values for k and x0. Table 2: Fitting results (along with standard errors) for fitting observed probabilities to mean leakage volumes Distance k (µ ± 1σ) x0 (µ ± 1σ) Adj. R2 SSD Experiments 1.5 m 7.8 ± 0.2 0.15 ± 0 1 3m 4.9 ± 0.3 0.47 ± 0.01 1 6m 6.2 ± 0.9 LLD Experiments

1.02 ± 0.02

0.98

6m

1.25 ± 0.04

0.99

4.0 ± 0.4


Page 8 of 18

Page 9 of 18


9m 12 m 15 m 242 243 244 245 246 247 248 249 250 251 252 253 254

255 256 257 258 259 260 261

4.7 ± 0.6 8.7 ± 2.2 6.2 ± 1.0

1.71 ± 0.04 2.11 ± 0.03 2.18 ± 0.02

0.98 0.92 0.95

Using the resulting 1000 realizations of fitted parameters of k and x0, we solve for each distance the value of x (leakage volume) that gives f = 0.5 (50% probability of detection, also median detection limit) and f = 0.9 (90% probability of detection). These results can be then plotted as a function of distance (Figure 3). In this case, distance for each realization is chosen from a normal distribution with the mean equal to the normal distance measure and standard deviation of  = 0.1 m (representing imprecision/variation in foot placement and camera position). The relationship between distance and required size of leak for each detection probability was fitted with linear, polynomial (order 2, 3, 4), exponential, logarithmic, and power law functions. Power law and quadratic functions fit the data with R2 > 0.8, as seen in nearly linear relationship in loglog space. We include power law fitting coefficients and fitted power law curves in Figure 3.

Figure 3. Size of leak as a function of distance where the probability of detection is 50% (red) or 90% (blue). Best power law fit shown in equations with fitted parameters. Adj. R2 for power law fits are R2 = 0.78 (90% prob) and R2 = 0.93 (50% prob) Interestingly, the size of leak required for detection grows as nearly the square of distance from leak. From an imaging perspective, the apparent size of an object in pixels (e.g., plume in this



262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299

case) is inversely proportional to the square of the imaging distance: the farther you image from, the smaller the image is. Thus, if plume size is approximately proportional to volume, one might conclude that the median detection limit would increase as the square of imaging distance. Discussion For an imaging distance of 3 m (10 ft, typical in field conditions), we estimate the 90% detection probability limit to be about 20 g/h. EPA, as part of recent methane emission regulation [16], noted that any leak detection technology should be able to detect 30 g/h of a 50-50 methanepropane mixture. EPA did not specify a detection probability at this leak rate. Assuming that EPA expects near-certain detection at 30 g/h (90% detection probability), a maximum line-of-sight imaging distance between 3 and 6 m (with reasonable exceptions) will be effective at capturing leak rates above the 30 g/h limit. Indeed, many field studies using OGI based cameras routinely find leaks in the 10 – 100 g/h range (see SI section S5), agreeing with our measured experimental values [3, 4]. Other estimates of detection limits in the literature can be found in Benson et al. [21]. Benson et al. use indoor experimental facilities to detect methane and other species using the FLIR GasFindIR (a predecessor to our camera). With a 25 mm focal length lens (similar to our camera), they find minimum detection limits of 0.8, 1.4, and 4.0 g CH4/h at distances of 3, 6 and 12 m, respectively (Table 5 in [21]). At these distances, we see 90% probability of detection at leak sizes more than an order of magnitude higher than those found by Benson et al. The discrepancy between this study and Benson et al.’s is that our outdoor field-based experiment had an everchanging imaging background (e.g., equipment type, weather, etc.), and leak configurations that are similar to natural gas facilities. Carefully controlled indoor conditions of Benson et al., produced an optimistic estimate of minimum detectable leak rate. Little information on detection probability exists for measurements at field sites. A few recent studies involving Picarro’s mobile flux plane technique [24], and Jet Propulsion Laboratory’s AVIRIS-NG technology [25] have attempted to experimentally measure detection probability curves. Most experimental studies of leakage at natural gas facilities do not have enough detailed information to determine detection probability curves. Therefore, true leak size distributions cannot be determined empirically. Observed leak-size distributions are convolutions of the true distribution and the detection probability curves of the specific leak-detection technology. However, a recent study in the Barnett shale region in Texas – the ERG Fort Worth Air Quality Survey (henceforth FWAQS) – used two leak detection methods to survey production facilities – EPA Method 21 (M21) and OGI [3]. Method 21 uses a concentration measurement device that can detect leaks with concentration signatures as small as 10 parts per million (ppm). Therefore, we can reasonably assume leak populations derived from a Method-21 survey to represent


Page 10 of 18

Page 11 of 18

300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319

324 325 326 327 328 329 330 331 332 333 334 335 336 337


approximately the true leak populations. With this assumption, we can estimate the detection probability curve of the OGI device as used in the survey. The FWAQS methodology used two separate teams to detect leaks: (1) one team used IR cameras to survey all components at sites and quantified and reported the observed leaks, and (2) the second team used M21 to survey 1 out of every 10 components, and quantified them. M21 surveys examine components at close proximity, classifying components as leaking if the local concentration is above a screening threshold (500 ppm). Because M21 is more sensitive than IR, we can use M21 results to estimate the “expected total” population of leaks that the IR team had the potential to find. A total of 1324 leaks are reported in the FWAQS dataset with an IR video time stamp, and are thus assumed to be found with IR. A total of 794 leaks are presented without an associated IR video file or video time stamp, and were thus assumed to be found via M21 only. Since 794 M21 leaks were found by surveying 1 out of 10 components, extrapolation suggests that the total number of M21 leaks at the sites studied was likely ten times as large (i.e.,  8000 leaks, subject to sampling noise). We will call this scaled population the “estimated total” leaks below. The size distributions of each population (scaledM21 and OGI) are presented as histograms in Figure 4. Table 3: Analysis of scaled FWAQS survey data based on two methods of leak detection – OGI and M-21 320 OGI (Yes) OGI (No) 321 M-21 (Yes) a (≈ 0) b (≈ 8000) 322 M-21 (No) c (1324) d (≈ 700K – 8K = 692K) 323 Table 3 shows the number of leaks detected in the FWAQS survey based on detection method. There is some ambiguity in the FWAQS survey about the overlap between the M21 and IR datasets. The total number of IR-found leaks is 1324 i.e., a + c = 1324 (see Table 3). IR-found leaks were later surveyed with M21 equipment, but it is unclear how many of these leaks were found originally with the M21 survey. We assume that all leaks with an IR time stamp were found only first via IR, then surveyed via M21, i.e., a ≈ 0 and c = 1324 (because only 10% of components were initially surveyed with M-21). Based on the prior assumption that leaks found with an M-21 survey represent true population, the scaled M-21 leak population (≈ 8000) will include all leaks at the facility, i.e., a + b = 8000. After converting reported leak volumes to gCH4/h (1.1984 mol/scf, 86.7 mol% CH4) we bin leaks found with M21 and IR into logarithmic bins from 10-2 to 104 gCH4/h, with bin width of 100.25 (i.e., bin 1:  10-2, < 10-1.75), then we scale M21 leaks by our expected total ratio (10x). We then compare the actual number of IR-found leaks in each bin to the expected total leaks in that bin.



338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355

356 357 358 359 360 361

If more IR leaks were found in a bin than estimated total leaks, the detection fraction is set to maximum of 100% (uncommon). This then gives us a percentage of the expected total leaks in each size bin that were found by IR camera (solid red line in Figure 4). Fitting to the same sigmoidal cumulative probability curve as above, we get k = 3.515 and x0 = 1.929. Because x0 is the location where f = 0.5, we expect 50% of true leaks to be found at x = 101.929 g CH4/h, or x = 84.9 g CH4/h, somewhat higher than the median detection limit estimated from our experiments (approx. 20 gCH4/h) (f is plotted as dashed red line in Figure 4). Because of the ambiguity in the overlap between different detection technologies, our “estimated total” leak population is likely a lower bound on the total number of leaks, and therefore our fraction detection is likely an upper bound. The higher limit could also be that compared to our experiments where the leak location was known, the FWAQS was looking at real facilities with unknown leak locations, perhaps making detection of small leaks more problematic. A final likely reason for the discrepancy between our measurements and the FWAQS could be ‘human factors’ – experience and skill level of the operator conducting the OGI survey. Future studies of on effectiveness of methane leak detection protocols should ideally involve multiple teams and different instruments measuring the same set of leaks across the entire facility.

Figure 4: Empirical results from ERG Fort Worth Air Quality Survey, n = 2118 leaks. Empirical distributions shown in semi-transparent histograms – the blue histogram corresponds to scaled leaks found using M-21, while the red histogram corresponds to leaks found using OGI. Fraction of expected total leaks found via IR, estimated by assuming leaks found using M-21 gives the true leak population, plotted as red line. Best-fit sigmoid to this detection probability is shown as red


Page 12 of 18

Page 13 of 18

362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399


dotted line. Large spike in histograms is a result of limitations of the quantification device, the Bacharach Hi-Flow sampler. Policy Implications The EPA, in its recent regulations to mitigate methane emissions, suggests that leak detection surveys be done with optical gas imaging technology (like the FLIR camera used here) or Method21. For newer technologies, EPA set a technology equivalence metric of minimum leak detection threshold at 30 g CH4/h for a 50-50 methane-propane gas coming out of a ¼” orifice [19]. In this study, we demonstrate that typical 90% leak detection threshold of optical gas imaging is about 20 g CH4/h at an imaging distance of 3 m, in line with EPA recommendations. However, it is not clear if higher sensitivity (lower 90% detection threshold) would lead to increased practical emissions mitigation. Here, we simulate marginal detection effectiveness as the sensitivity of the detection technology improves. We use the median detection threshold leak-rate as a proxy for sensitivity – higher values imply lower sensitivity and vice-versa. We use publicly available data sets from production well-sites, and compressor stations at gathering and boosting, transmission, and storage facilities to estimate the fraction of emissions detected as a function of the median detection threshold (see Figure 5). We use the Fugitive Emissions Abatement Simulation Toolkit (FEAST) open-source framework to simulate the timeevolution of leaks at natural gas facilities [26]. Periodic LDAR surveys are simulated to be conducted semi-annually at production well-pads and quarterly at compressor stations. The detection probability curves are modeled as a sigmoid with the median detection threshold as the ‘µ’ parameter and slope parameter set to 1. The facilities are simulated for a total of 8 years, with the baseline emissions at production and gathering and boosting sites set at 5 tons per year (tpy) and 37 tpy, respectively, according to EPA assumptions [19]. Furthermore, default values for average component and equipment count at each of these facilities is derived from the Background Technical Support Documentation provided by the EPA [19]. Figure 5 shows the total emissions detected at the end of 8 years at production well-pads (blue) and gathering and boosting compressor stations (red) as a function of the median detection threshold – note that the x-axis is reversed from scales above. In both facility types shown, the fraction of emissions detected saturates at a median detection limit of ≤100 g CH4/h. Any improvements in sensitivity beyond this limit does not result in a corresponding improvement in total emissions detected. This saturation median detection threshold (100 g CH4/h) is significantly higher than EPA requirements for demonstrating technology equivalence (90% probability of detection at 30 g CH4/h). This is a direct result of the highly skewed leak-size distributions observed at natural gas facilities [14] – ‘super-emitters’ account for a significant fraction of the



400 401 402 403 404 405 406 407 408 409

410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426

total emissions. These ‘super-emitters’ can be easily detected with a technology that has lowersensitivity. Any additional improvement in sensitivity only results in the detection of smaller leaks that do not contribute significantly to the total emissions detected – hence the observed saturation in Figure 5. It is worth noting that even for very high sensitivity systems (e.g., median detection limit = 100 g CH4/h), all leakage is not mitigated because of the dynamic nature of leakage and the periodic survey design. Clearly, requiring higher sensitivity from newer technologies can potentially increase costs without corresponding mitigation benefits. Similar analysis for compressor stations in the transmission and storage sectors are given in the SI section S5.

Figure 5: Fraction of total emissions mitigated at production well-pads (blue) and gathering and boosting compressor stations (red) through semi-annual and quarterly LDAR surveys, respectively, as a function of the median detection limit of the leak-detection technology. EPA expected mitigation levels are shown as colored dotted lines, while the EPA detection limit for demonstrating technology equivalence (30 g/h) is shown as a black dotted line. In both cases, the fraction of emissions detected saturates at a median detection limit of about 200 g CH4/h. The real-world marginal mitigation effectiveness as a function of detection sensitivity is zero beyond certain detection limits. For example, a median detection limit of 100 g CH4/h is sufficient to capture the maximum amount of leakage that is possible through semi-annual or quarterly LDAR surveys. Given that OGI exceeds this threshold by a factor of 5 when imaging from 3 m, we conclude that OGI technology is ‘good enough’ for methane leak detection, and that higher sensitivity is not necessary. In other words, given a periodic survey design, even a hypothetical perfect leak detection system where all leaks are detected cannot achieve 100% mitigation. Increasing the minimum detection limits for technology equivalence demonstrations may help reduce detection costs and further make LDAR programs attractive to operators. Policy makers


Page 14 of 18

Page 15 of 18


427 428 429 430 431 432 433 434 435 436 437 438

should carefully consider the results from this study to design effective protocols to approve newer and likely cheaper leak detection technologies. More work is required to realistically compare benefits and costs of very different technologies with different detection thresholds, like aerial/drone-mounted detection options which many be less sensitive but significantly faster at surveying large areas compared to OGI systems.

439 440

References

Supporting Information: Details on flow rate calibration, uncertainty analysis, and additional information on policy analyzes.

[1] U.S. Energy Information Agency, "Short Term Energy Outlook," pp. 10-11, July 2016. [2] X. Zhang, N. P. Myhrvold and K. Caldiera, "Key factors for assessing climate benefits of natural gas versus coal electricity generation," Environ. Res. Lett. 2014, 9, 114022. [3] City of Fort Worth, "Natural Gas Air Quality Study Technical Report," 2011. [4] D. Allen, V. Torres, J. Thomas, D. Sullivan, M. Harrison, A. Hendler, S. Herndon, C. Kolb, M. Fraser, A. Hill, B. Lamb, J. Miskimins, R. Sawyer and J. Seinfeld, "Measurements of methane emissions at natural gas production sites in the United States," Proc. Natl. Acad. Sci. 2013, 110, 17768-17773. [5] J. Kuo, T. C. Hicks, B. Drake and T. F. Chan, "Estimation of methane emission from California natural gas industry," J. Air. Waste Manag. Assoc. 2015, 65, 844-855. [6] A. J. Marchese, T. Vaughn, D. Zimmerle, D. Martinez, L. Williams, A. Robinson, A. Mitchell, R. Subramanian, D. Tkacik, J. Roscioli and S. Herndon, "Methane emissions from United States natural gas gathering and processing," Environ. Sci. Technol. 2015, 49, 1071810727. [7] D. Zimmerle, L. Williams, T. Vaughn, C. Quinn, R. Subramanian, G. Duggan, B. Willson, J. Opsomer, A. Marchese, D. Martinez and A. Robinson, "Methane emissions from the natural gas transmission and storage system in the United States," Environ. Sci. Technol. 2015, 49, 9374-9383. [8] K. McKain, A. Down, S. Raciti, J. Budney, L. Hutyra, C. Floerchinger, S. Herndon, T. Nehrkorn, M. Zahniser, R. Jackson, N. Phillips and S. Wofsy, "Methane emissions from natural gas infrastructure and use in the urban region of Boston, Massachusetts," Proc. Natl. Acad. Sci. 2014, 112, 1941-1946. [9] R. B. Jackson, A. Down, N. G. Phillips, R. C. Ackley, C. W. Cook, D. L. Plata and K. Zhao, "Natural gas pipeline leaks across Washington, DC," Environ. Sci. Technol. 2014, 48, 20512058.



[10] T. Lavoie, P. Shepson, C. Gore, B. Stirm, R. Kaeser, B. Wulle, D. Lyon and J. Rudek, "Assessing the methane emissions from natural gas-fired power plants and oil refineries," Environ. Sci. Technol. 2017 51, 3373-3381. [11] A. Brandt, G. Heath, E. Kort, F. O'Sullivan, G. Petron, S. Jordaan, P. Tans, J. Wilcox, A. Gopstein, D. Arent, S. Wofsy, N. Brown, R. Bradley, G. Stucky, D. Eardley and R. Harriss, "Methane leaks from North American natural gas systems," Science 2014, 343, 733-735. [12] US Environmental Protection Agency, Inventory of US Greenhouse Gas Emissions and Sinks: 1990-2015, 2017. [13] S. Schwietzke, G. Petron, S. Conley, C. Pickering, I. Mielke-Maday, E. Dlugokencky, P. Tans, T. Vaughn, C. Bell, D. Zimmerle, S. Wolder, C. King, A. White, T. Coleman, L. Bianco and R. Schnell, "Improved mechanistic understanding of natural gas methane emissions from spatially resolved aircraft measurements," Environ. Sci. Technol. 2017, 51, 7286-7294. [14] A. R. Brandt, G. A. Heath and D. Cooley, "Methane leaks from natural gas systems follow extreme distributions," Environ. Sci. Technol. 2016, 50, 12512-12520. [15] White House, "North American Climate, Clean Energy, and Environment Partnership Action Plan," Office of the Press Secretary, Washington, DC, 2016. [16] US Environmental Protection Agency, "New source performance standards; oil and natural gas sector: Emission standards for new, reconstructed, and modified sources," Fed. Reg. 2016, 81, 35824-35942. [17] Keating Research Inc., "The Colorado case study on methane emissions: conversations with the oil and gas industry," Telluride, CO, 2016. [18] A. P. Ravikumar, J. Wang and A. R. Brandt, "Are optical gas imaging techniques effective for methane leak detection?," Environ. Sci. Technol. 2017, 51, 718-724. [19] US Environmental Protection Agency, "Background Technical Support Document for the Proposed New Source Performance Standsards 40 CFR Part 60, subpart OOOOa," 2015. [20] A. P. Ravikumar and A. R. Brandt, "Designing better methane mitigation policies: the challenge of distributed small sources in the natural gas sector," Env. Res. Lett. 2017, 12, 040023. [21] R. Benson, R. Madding, R. Lucier, J. Lyons and P. Czerepuszko, "Standoff passive optical leak detection of volatile organic compounds using a cooled InSb based infrared imager," in AWMA 99th Annual Meeting Papers, New Orleans, LA, 2006. [22] FLIR Systems Inc., "FLIR Gas-Find 320," Wilsonville, OR, 2011. [23] National Gas Machinery Laboratory; Clearstone Engineering Ltd.; Innovative Environmental Solutions, "Cost-effective directed inspection and maintenance control opportunities at five gas processing plants and upstream gathering compressor stations and well sites: EPA Phase II Aggregate Site Report," 2006. [24] C. W. Rella, T. R. Tsai, C. G. Botkin, E. R. Crosson and D. Steele, "Measuring Emissions from Oil and Natural Gas Well Pads Using the Mobile Flux Plane Technique," Environ. Sci. & Technol. 2015, 49 (7), 4742-4748.


Page 16 of 18

Page 17 of 18


[25] A. K. Thorpe, C. Frankenberg, A. D. Aubrey, D. A. Roberts, A. A. Nottrott, T. A. Rahn, J. A. Sauer, M. K. Dubey, K. R. Costigan, C. Arata, A. M. Steffke, S. Hills, C. Haselwimmer, D. Charlesworth, C. C. Funk, R. O. Green, S. R. Lundeen, J. W. Boardman, M. L. Eastwood, C. M. Sarture, S. H. Nolte, I. B. Mccubbin, D. R. Thompson and J. P. McFadden, "Mapping methane concentrations from a controlled release experiment using the next generation airborne visible/infrared imaging spectrometer (AVIRIS-NG)," Remote Sensing of Environ. 2016, 179, 104-115. [26] C. E. Kemp, A. P. Ravikumar and A. R. Brandt, "Comparing natural gas leakage detection technologies using an open-source virtual gas field simulator," Environ. Sci. Technol. 2016 50, 4546-4553. [27] J. A. Littlefield, J. Marriott, G. A. Schivley and T. J. Skone, "Synthesis of recent ground-level methane emission measurements from the US natural gas supply chain," J. Cleaner Prod. 2017, 148, 118-126.



TOC art 71x47mm (300 x 300 DPI)


Page 18 of 18

Good versus Good Enough? Empirical tests of methane leak detection

Recommend Documents