Assessing the Reliability of Material Flow Analysis Results: The Cases

Subscriber access provided by University of Sussex Library

Article

Assessing the reliability of MFA results: The cases of rhenium, gallium, and germanium in the US economy Grégoire Meylan, Barbara K. Reck, Helmut Rechberger, Thomas E. Graedel, and Oliver Schwab Environ. Sci. Technol., Just Accepted Manuscript • DOI: 10.1021/acs.est.7b03086 • Publication Date (Web): 18 Sep 2017 Downloaded from http://pubs.acs.org on September 22, 2017

Just Accepted “Just Accepted” manuscripts have been peer-reviewed and accepted for publication. They are posted online prior to technical editing, formatting for publication and author proofing. The American Chemical Society provides “Just Accepted” as a free service to the research community to expedite the dissemination of scientific material as soon as possible after acceptance. “Just Accepted” manuscripts appear in full in PDF format accompanied by an HTML abstract. “Just Accepted” manuscripts have been fully peer reviewed, but should not be considered the official version of record. They are accessible to all readers and citable by the Digital Object Identifier (DOI®). “Just Accepted” is an optional service offered to authors. Therefore, the “Just Accepted” Web site may not include all articles that will be published in the journal. After a manuscript is technically edited and formatted, it will be removed from the “Just Accepted” Web site and published as an ASAP article. Note that technical editing may introduce minor changes to the manuscript text and/or graphics which could affect content, and all legal disclaimers and ethical guidelines that apply to the journal pertain. ACS cannot be held responsible for errors or consequences arising from the use of information contained in these “Just Accepted” manuscripts.

Environmental Science & Technology is published by the American Chemical Society. 1155 Sixteenth Street N.W., Washington, DC 20036 Published by American Chemical Society. Copyright © American Chemical Society. However, no copyright claim is made to original U.S. Government works, or works produced by employees of any Commonwealth realm Crown government in the course of their duties.

Page 1 of 27

Environmental Science & Technology

1 2

Assessing the reliability of MFA results: The cases of rhenium, gallium, and germanium in the US economy

3

Grégoire Meylan1,2,*, Barbara K. Reck3, Helmut Rechberger4, Thomas E. Graedel3, Oliver

4

Schwab4

5

1

6

Zurich, Switzerland

7

2

8

ETH Zurich, 8093 Zurich, Switzerland

9

3

Transdisciplinarity Lab, Department of Environmental Systems Science, ETH Zurich, 8092

Safety and Environmental Technology Group, Institute for Chemical and Bioengineering,

Center for Industrial Ecology, Yale School of Forestry and Environmental Studies, Yale

10

University, New Haven 06511, USA

11

4

12

Austria

13

*Corresponding author: ETH Zurich, USYS TdLab, CHN K 76.2, Universitaetstrasse 22,

14

8092 Zurich, Switzerland, [email protected]

Institute for Water Quality, Resource and Waste Management, TU Wien, 1040 Vienna,

15

1 ACS Paragon Plus Environment


16

Table of Contents (TOC)/Abstract Art A priori information Missing information

Information by balancing

Ge Ga Re 0

17

100

200

300

Informational units

18


Page 2 of 27

Page 3 of 27


19

Abstract

20

Decision-makers traditionally expect “hard facts” from scientific inquiry, an expectation the

21

results of Material Flow Analyses (MFAs) can hardly meet. MFA limitations are attributable

22

to incompleteness of flowcharts, limited data quality, and model assumptions. Moreover,

23

MFA results are mostly less based on empirical observation and rather on social knowledge

24

construction processes. Developing, applying, and improving means for evaluating and

25

communicating reliability of MFA results is imperative. We apply two recently proposed

26

approaches for making quantitative statements on MFA reliability to national minor metals

27

systems: rhenium, gallium, and germanium in the US in 2012. We discuss the reliability

28

results in policy and management contexts. The first approach consists in assessing data

29

quality based on systematic characterization of MFA data and associated meta-information

30

and quantifying the “information content” of MFAs. The second is a quantification of data

31

inconsistencies indicated by the “degree of data reconciliation” between data and model.

32

Increasing information content and decreasing degree of reconciliation indicate reliable or

33

certain MFA results. This article contributes to reliability and uncertainty discourses in MFA,

34

exemplifying the usefulness of the approaches in policy and management, and to raw material

35

supply discussions by providing country-level information on three important minor metals

36

often considered as critical.

37

Keywords: Material Flow Analysis, data quality, uncertainty, information content, data

38

reconciliation, minor metals, decision making, rhenium, gallium, germanium

39

Introduction

40

Material Flow Analysis (MFA) can be applied as a means for modeling scenarios, for

41

comparing systems of alternative design, and for making forecasts regarding natural or

42

anthropogenic material flow systems. Such applications of MFA provide relevant information

43

for supporting decision-making processes for example regarding the management of resource 3 ACS Paragon Plus Environment


44

systems. Other than that, MFA is widespread as a reference procedure facilitating material

45

accounting activities. The goal of such activities is to map physical realities as flowcharts of

46

arbitrary level of detail visualizing flows between processes in regions and system imports

47

and exports. Such representations of real-world systems follow a quasi-standardized three-

48

step procedure. In the first step, a qualitative system is designed. Its structure reflects all

49

relevant flows and processes of the associated real-world system. In the second step, data (or

50

a priori information) for quantifying this qualitative system are collected. In the third step, the

51

data are balanced in order to satisfy mass balance constraints, to erase discrepancies and to

52

close data gaps, and thus to create new information (a posteriori information - after

53

application of the MFA procedures) on a material flow system.

54

Decision-makers typically expect “hard facts” from scientific inquiry1. However, MFA results

55

are only partly based on empirical observation but rather on collective knowledge

56

construction processes put forward by MFA communities. Consequently, a characteristic

57

especially typical of accounting MFAs is that they entail certain degrees of subjectivity and

58

arbitrariness, which may arise in all three of the above mentioned steps of the analysis

59

procedure: It concerns the design of system structure (step one, cf. Klinglmair, et al. 2), the

60

collection, integration, and evaluation of data (step two, cf. Laner, et al. 3), and also the

61

treatment of data in the balancing model which contains possibly strong assumptions (step

62

three, cf. Cencic 4). As a consequence, MFA findings may not satisfy the initial expectation of

63

decision-makers to face “hard facts”.

64

Acknowledging the necessity to evaluate and communicate the reliability of MFA results in

65

policy and management, we address the impact of the incompleteness of balance entries or the

66

inaccuracy of utilized information (i.e., step 2) in the field of minor metal MFAs. The latter is

67

receiving increasing scrutiny in the context of critical material debates5-7. Lack of information

68

and unclear reliability of results are long-known issues in MFA. Nevertheless, systematic 4 ACS Paragon Plus Environment

Page 4 of 27

Page 5 of 27


69

procedures for their evaluation and communication which acknowledge the specific nature of

70

given information are rather recent. As a contribution to developments in this regard, the goal

71

of the study is two-fold. First, we demonstrate the feasibility of two approaches for reliability

72

assessment in this particular field and suggest recommendations for application and

73

improvement. Second, we compare both approaches and identify comparative advantages of

74

each approach for minor metal MFAs. The impact of information shortcomings will be

75

addressed in this article regarding three case studies of minor metals in the US economy,

76

which are introduced in the following.

77

Materials and Methods

78

Minor metals in the US

79

Minor metals are metals not traded on the London Metal Exchange and often mined as co- or

80

by-products of major metals, the latter then also called “host metals”8, 9. Minor metals are

81

vital to developed and emerging economies not only with regard to emerging environmental

82

technologies, but also more conventional application fields10,

83

rhenium are often considered as critical metals5, 6, explained by their importance in modern

84

technology12-14 in combination with a somewhat inflexible supply, a result of them being

85

primarily mined as by-products8, 15. While gallium and germanium are important mainly to

86

information technology and communication, rhenium is a key element in superalloys utilized

87

in the aerospace industry. Both these sectors are of key importance to modern economies. In

88

addition to air transportation, space exploration relies on nickel-based superalloys containing

89

rhenium to increase the heat resistance of launchers16. These are just some instances of

90

important technological applications of minor metals.

91

In the Supporting Information SI1, the life cycle stages Production, Fabrication &

92

Manufacturing, Use, and Waste Management & Recycling of the three metals, including

93

stocks and flows, are described in more detail based on information from the U.S. Geological 5 ACS Paragon Plus Environment

11

. Gallium, germanium, and


Page 6 of 27

94

Survey, scientific reports and expert estimations. We also present corresponding qualitative

95

MFA systems of these metals. The differences in level of detail and in the quality of databases

96

result in varying degrees of knowledge on the analyzed systems. While practitioners

97

composing MFAs may have a good intuitive understanding of the quality of their analysis

98

results, it is not clear how this should be systematically evaluated in a transparent,

99

reproducible way and how this information should be communicated in an accessible way to

100

decision-makers. Approaches to this shortcoming are addressed in the following.

101

Assessing and comparing the reliability of MFA results

102

Both the design of qualitative MFAs and the quality of MFA data are factors potentially

103

limiting the reliability of MFA results. The evaluation and representation of system

104

uncertainty arising from the MFA practitioner’s knowledge gaps in flowchart design is a

105

typically intricate task. Flowchart design is based on the best knowledge available to the

106

composing agent who consequently could improve the chosen system design only after better

107

information becomes available. Analytical approaches for identifying possible design

108

problems are confined to the comparison of systems to reference systems with presumably

109

similar structures (possibilities for formal quantitative evaluation of system structures in MFA

110

and related methods have been proposed, for example, by Navarrete-Gutiérrez, et al.

111

Layton, et al. 18 and Schwab and Rechberger 19). System uncertainty arising from ignorance in

112

system design is not addressed in this article but is considered elsewhere2.

113

System uncertainty arising from data quality limitations and its effect on the reliability of

114

MFA results dominate the discourse on uncertainty in MFA3,

115

uncertainty in indium, gallium and germanium systems has been discussed qualitatively, see

116

e.g., Licht, et al.

117

MFA are often hindered by the fact that MFA data are typically isolated values and not

118

datasets. Having been identified as a mainly “epistemic” phenomenon (epistemic uncertainty

17

,

20, 21

. For example, the

14

. Quantitative statistical procedures for assessment of data uncertainty in


Page 7 of 27


119

– uncertainty due to lack of knowledge), data uncertainty in MFA appears to be a non-

120

stochastic problem and thus evades statistical approaches (cf. Laner, et al. 3). As empirical

121

evidence on flow data uncertainty is virtually always absent, the popular application of

122

estimated or assumed uncertainty ranges, percentages, and distributions may raise the

123

unjustified impression of empirical evidence. Moreover, due to the fact that qualitative

124

systems of the same metal, the same year, and the same spatial unit will most likely differ to a

125

certain extent because of author choices and the goal and scope of analyses, comparing the

126

uncertainty of individual flows is often not possible. Instead of focusing on the uncertainty of

127

individual flows, providing uncertainty information for a material flow system as a whole

128

opens new perspectives regarding assessability and comparability of MFA results and enables

129

answering intriguing research questions. For instance, are MFA results for a given metal and

130

spatial focus becoming more reliable over time? Also, it is generally accepted that systems of

131

base metals like iron and aluminum are better known than those of minor metals or rare

132

earths, the usage of which is more recent and concentrated in a rather small number of firms,

133

so that data are often withheld. But how much better are they known? Particularly in light of

134

considerably limited information, as holds for the minor metal systems analyzed in this

135

article, the need for adequate approaches to evaluate and communicate uncertainty in MFA

136

systems is apparent. Two possibilities are introduced in the following.

137

Two approaches to evaluate reliability in MFA

138

MFA practitioners have possibilities for evaluating, analyzing, and communicating the

139

reliability of MFA results at several stages in the analysis process, including the two

140

following: First, during the data collection process, they can assign reliability indicators to

141

collected data, reflecting their degree of belief that data are true in a given context. A low

142

degree of belief suggests the existence of potentially unreliable MFA results. And second,

143

MFA practitioners can observe the degree of adjustment the collected data undergo during the 7 ACS Paragon Plus Environment


144

reconciliation process. A lower degree of data reconciliation reflects little discrepancies and

145

thus hints at higher reliability. These two approaches to evaluate reliability in MFA are

146

applied and discussed in this article.

147

Degree of belief in MFA data to be true

148

Ravetz

149

uncertainty in policy-oriented science. Building on this seminal work, Schwab and

150

colleagues24 propose a procedure for systematically characterizing MFA data by a set of pre-

151

defined criteria, which are later aggregated to quantitative judgments on the accuracy of

152

information in given MFA systems. The approach aims at quantifying the “information

153

content” of material flow systems and its counterpart, the information missing to fully

154

understand a system (uncertainty) of a given structure. As a basis to this evaluation, MFA data

155

are, in a first step, systematically characterized by so-called data attributes as documented in a

156

data characterization matrix (DCM)24. The DCM allows for a detailed documentation of data

157

used in an MFA and associated meta-information. In the second step, data collected to

158

quantify flow variables are evaluated based on the data attributes. The attribute scores are

159

aggregated to data quality measures designated as „information defects“ (ID, Schwab, et al.

160

25

161

equal to or in the vicinity of 1) designate data of poor quality. IDs are specific to any flow in

162

its particular system context. In the third step, the aforementioned information content of

163

MFA systems is calculated as the difference between the system uncertainty U at different

164

stages in the modelling process (Uap, Ub, Ub,w see Eq. 1-3 and corresponding description

165

below) and the maximum system uncertainty (S, see Eq. 4). The calculation integrates the IDs

166

as variables for data quality and takes advantage of mathematical features of the information

167

entropy equation for calculating the system information measures. The derivation of Eq. 1-4 is

168

provided in Schwab and Rechberger 19.

22

and Funtowicz and Ravetz

23

developed ways to manage, assess, and communicate

). Low IDs (ID equal to or in the vicinity of 0) designate data of good quality, high IDs (ID


Page 8 of 27

Page 9 of 27


∑

Eq. 1

, ∑ ,

Eq. 2

= − log

= − , log

, = −

, , , log ∑ , ∑ , = − log

1

Eq. 3

Eq. 4

169 170

In Eq. 1-4, nF is the number of flow variables in a system, IDFi is the information defect of

171

flow Fi (IDFi,b after balancing) and XFi,b is the quantity of the balanced flow variable Fi. Three

172

points of evaluation relate to the system uncertainty after balancing (Ub), which is typically

173

lower than the uncertainty of a system with a priori information (Uap) and the maximum

174

uncertainty of a system, that is, a system without data (S, for “informational” system size,

175

measured in “informational units”). S also defines the potential information content of a

176

system in which each flow is known with absolute certainty. Ub and Uap resemble the system

177

uncertainty or the “amount of missing information”. The difference between the actual system

178

uncertainty (Uap or Ub) and the system size S is referred to as the information content of a

179

material flow system. An additional point of evaluation is Ub,w, in which the uncertainty of

180

each flow after balancing is additionally weighted by its quantitative relevance in a system

181

(∑, ). This is to reflect that, in order to understand a material flow system, knowing

182

quantitatively major flows better is more important than knowing quantitatively minor flows.

183

Degree of data reconciliation

184

In a typical MFA procedure, data is collected to quantify as much flows of a system as

185

possible. Diverging data – a priori information – are manipulated so that mass balance

186

constraints are fulfilled. The degree to which prior data are manipulated increases with

,



26

Page 10 of 27

187

discrepancies in the system. Zoboli, et al.

188

measure for identifying potential errors and for evaluating the reliability of MFA results. This

189

approach was discussed and formally specified by Klinglmair, et al. 2. Accordingly, the effect

190

of data reconciliation is quantified as the mean relative deviation of the reconciled data

191

from the entered value of flow Fi (Eq. 5).

=

∑ |

interpret the degree of data reconciliation as a

− | × 100

Eq. 5

192

The higher , the more the available data are inconsistent within the predefined MFA system.

193

Decreasing indicate decreasing reconciliation and thus hint at consistent systems and

194

consequently on results of increasing reliability.

195

These two approaches, the information content as calculated from the degree of belief in MFA

196

data and the degree of data reconciliation as calculated from the deviation of balanced data

197

from a priori data, are chosen for analysis of reliability and uncertainty. We exemplify the

198

usefulness of such global information on the reliability of MFA results for decision-making

199

processes in policy and management with the cases of rhenium, gallium, and germanium in

200

the US in 2012 introduced earlier.

201

Results and Discussion

202

Results of the Material Flow Analyses

203

We prepared the quantitative flowcharts presented in Figures 1-3 using the software STAN

204

(www.stan2web.net27). A documentation of system design and all applied data is provided in

205

the appendices SI1 (system design), SI2 (rhenium), SI3 (gallium) and SI4 (germanium).


Page 11 of 27


206 207 208 209 210 211

Figure 1: Flowchart of the 2012 US rhenium system. Flow quantities (in Mg [metric tons per year]) are rounded to two significant digits (APR: ammonium perrhenate, Dissipated Re: rhenium dissipated by petroleum-reforming catalysts while in use, ENG: Engines, F&Mfg: Fabrication & Manufacturing, MISC: miscellaneous, PETRO: petroleum-reforming catalysts, Re metal: rhenium metal, U: Use, WM&R: Waste Management & Recycling).

212 213

The import of ammonium perrhenate (APR) and rhenium metal to the metal market represents

214

the largest flow of the 2012 US rhenium system. This flow also happens to be the largest

215

source of rhenium into the system, being more than three times more important than rhenium

216

mined in the US or obtained on the scrap market. The largest rhenium sink is found in Use of

217

engines, in which most of rhenium entering is added to stock. Such a large addition to stock is

218

explained by the recent surge in the use of rhenium in engines. As the aerospace industry is

219

increasing its use of rhenium while developing and strengthening the reuse of rhenium-

220

containing superalloy scrap, one can assume that in the near future the process of Fabrication

221

& Manufacturing of superalloys will overshadow other rhenium applications and that the

222

superalloy urban mine will be efficiently exploited.



223 224 225 226 227 228

Figure 2: Flowchart of the 2012 US gallium system. Flow quantities (in Mg [metric tons per year]) are rounded to two significant digits (F&Mfg: Fabrication & Manufacturing, GaAs: gallium arsenide, GaN: gallium nitride, HPM: high-purity metal, IC: integrated circuits, IW: industrial waste, MISC: miscellaneous, OPTO: optoelectronic devices, U: Use, WM&R: Waste Management & Recycling).

229 230

The largest flow of the 2012 US gallium system is the import of low purity metal into primary

231

refining. As in the case of rhenium, imports are also the largest source feeding the system.

232

Foreign supplies are twice as important as new scrap fed to secondary refining. Exports of

233

high purity metal correspond to the largest sink. Generally speaking, processes belonging to

234

Production and Fabrication & Manufacturing, together with the scrap market, largely

235

dominate the gallium system because of high losses in Fabrication of gallium arsenide and

236

gallium nitride.


Page 12 of 27

Page 13 of 27


237 238 239 240 241

Figure 3: Flowchart of the 2012 US germanium system. Flow quantities (in Mg [metric tons per year]) are rounded to two significant digits (ELEC: electrical & electronic products, F&Mfg: Fabrication & Manufacturing, IR: infrared, IW: industrial waste, MISC: miscellaneous, MOC: metal, oxide, chloride, U: Use, WM: Waste Management).

242 243

The largest flow of the 2012 US germanium system is the import of the precursors of

244

Fabrication, to be processed into infrared systems, fiber optics, electrical/solar panels, and

245

other first uses. These imports consist of metals, dioxides, and tetrachlorides, and represent

246

the largest source into the system, albeit closely followed by US zinc mines. Landfills are the

247

largest sink, with residues from Recycling being the largest contributor. However, the amount

248

exported as concentrates and precursors of Fabrication surpasses the quantity of landfilled

249

germanium.

250

The first similarity between the three material systems is the same order of magnitude of their

251

flows. In all three systems, imports of raw materials or precursors of Fabrication represent the

252

largest flows, while imports of semi-fabricated or end-use products are quite small. Raw

253

material and precursor imports are also the main sources, while domestic mines and scrap

254

markets play different roles depending on the metal considered. The three systems present 13 ACS Paragon Plus Environment


255

different pictures in terms of the most important sinks. Yet, it is clear that landfills are still

256

important sinks and improvements are required at different levels: the anticipation of

257

collection of superalloy scrap in the case of rhenium and efficient end-of-life recycling in the

258

cases of gallium and germanium.

259

Reliability of the MFA results

260

Beyond the differences and similarities when it comes to material quantities, the three cases

261

differ in terms of the number and type of processes and flows, as well as in their databases.

262

General characteristics of the systems are provided in Table 1.

263


Page 14 of 27

Page 15 of 27

264 265


Table 1: General system characteristics of the analyzed rhenium (Re), gallium (Ga), and germanium (Ge) systems analyzed according to Schwab, et al. 24.

Quantity Database characteristic Re

Ga

Ge

No. of flow variables in system

40

42

48

No. of a priori unknown flow variables

7

5

4

number of processes

14

14

16

number of stocks

5

5

5

total no. of information elements

99

261

159

total no. of data elements

99

261

159

average no. of info. elements per flow

3.0

7.1

3.6

average no. of data elements per entity

1

1

1

32

19

21

100

100

100

flows described directly by autonomous data (%) isolated values (%) 266 267

According to Table 1, the systems differ when it comes to the number of data gaps, the share

268

of flows that can be described by ready-to-apply data, and the overall data collection effort.

269

As to the latter, 261 information elements were collected for quantifying 37 flows in the

270

gallium system and 99 information elements were collected for quantifying 33 flows in the

271

rhenium system. Information inventories including all data applied in the MFAs and a

272

transparent data characterization and data quality evaluation of each individual flow are

273

provided in appendices SI2-4. In combination with these data quality evaluations, Eq. 1-4

274

yield the results provided in Table 2.



275 276 277

Page 16 of 27

Table 2: System measures (System size S, system uncertainty U, and degree of data reconciliation D) of the three minor metal systems, analyzed according to Schwab and Rechberger 19 and Klinglmair, et al. 2 and rounded to the nearest integer (rhenium (Re), gallium (Ga) and germanium (Ge)).

278

Re

Ga

Ge

213

227

262

Uap

124

148

126

Ub

90

106

93

Ub.w

73

89

85

11.1

5.5

System size

S Uncertainty


(%)

6.2

279 280

The reliability metrics calculated according to Eq. 1-4 are provided in Table 2. The potential

281

uncertainty of the system (S) decreases to Uap after a priori information on flow quantities has

282

been implemented in the model, and to Ub after this information has been balanced. With

283

decreasing uncertainty, the information content of the MFAs increases, as illustrated in Figure

284

4. For example, in the gallium system the share of a priori information is, in relation to the

285

other two systems, small, and the information gained by balancing exceeds that gained by

286

balancing in the rhenium and germanium systems. Nevertheless, of the three studies, the

287

largest missing information occurs in the gallium system. One can summarize that of the three

288

systems the germanium MFA provides the most information, even though there still is a

289

considerable share of uncertainty remaining. Detailed information on each individual flows’

290

contribution to the system uncertainty is provided in appendices SI2-4.


Page 17 of 27


300 Missing Information Inform. by balancing

93

A priori information

33

106 79

92

137

100

42

34

90

informational units

200

0

291 292 293 294

Re

Ga

Ge

Figure 4: Graphical representation of the information content and the system uncertainty of the rhenium (Re), gallium (Ga), and germanium (Ge) systems. A priori information, information gained by balancing, and missing information are calculated from the values provided in Table 2.

295 296

A further metric indicating that the information basis of the germanium system is possibly the

297

most robust of the three cases is the degree of data reconciliation (Table 2). The results

298

indicate that the database of the gallium system is, with a degree of data reconciliation of

299

more than 11%, the least consistent, and that the information basis of the germanium system

300

is the most consistent with a of less than 6%.

301

Weighting the uncertainty remaining in the systems by the flow quantities (Ub,w in Table 2,

302

see Eq. 3) and relating the uncertainty to the system size S leads to the following results: the

303

rhenium system is known to 65%, the gallium system to 61%, and the germanium system to

304

68%. For comparison, according to Schwab and Rechberger 19 an aluminum MFA was found

305

to be known to 85% and a plastics MFA to 70%. This result illustrates that on the way to

306

providing more reliable information on minor metal flows in the US economy there is need

307

for further scientific inquiry and enhanced data provision efforts from authorities,

308

associations, and other involved stakeholders.



309

Discussion

310

The results of the complete minor metal systems help US industry associations and policy-

311

makers prioritize action for better management of minor, potentially critical metal systems.

312

They highlight the US systems’ dependency on raw materials and precursors of Fabrication,

313

while demonstrating that landfill deposits are accumulating quickly. Increasing end-of-life

314

recycling rates and thereby reducing import reliance will reduce the US’s criticality by

315

addressing its supply risk and its vulnerability to supply restriction, respectively28. It is clear

316

that the development of recycling technologies is not in the hands of single actors like

317

national industry associations or policy-makers, but are the result of complex interactions

318

between actor networks and the biophysical world as conceptualized by frameworks such as

319

the Technological Innovation System (TIS) framework29-31. Nonetheless, MFAs as knowledge

320

bases can become crucial inputs to TISs and significantly alter their trajectories towards more

321

efficient outcomes thanks to the MFA framework’s holistic perspective. To what extent

322

MFAs will make an impact in a TIS will ultimately depend on their reliability. Therefore,

323

policy-oriented MFAs should include an assessment of reliability such as the ones we

324

presented and applied in this article. In the following paragraphs, we discuss the application

325

of the two reliability assessment methods to the minor metal cases, starting with information

326

content and system uncertainty, and identify comparative advantages.

327

Application and recommendations

328

With respect to the system size S indicating the number of flows in the system and thus

329

describing maximum system uncertainty (Table 2), the three minor metal systems are smaller

330

than other MFA systems such as Aluminum (S=483), Plastics (S=568), or Phosphorus

331

(S=846) (see Schwab and Rechberger

332

analyzed minor metals are applied in fewer areas of the analyzed economies than many other

333

materials. However, the system size also depends on the resolution of economic sectors

19

). Certainly a reason for these differences is that the


Page 18 of 27

Page 19 of 27


334

adopted in an MFA. In the case of gallium, optoelectronic devices and integrated circuits are

335

groups of various products. More explicit modelling of the product systems would lead to a

336

much larger S. Hence, we recommend using the same resolution of economic activities to

337

ensure the comparability of MFAs of a metal across different regions or time horizons.

338

The systematic analysis of minor metal flows is only of comparatively recent interest in

339

scientific inquiry. Therefore, the understanding of the systems and the available information

340

basis is comparatively limited, which is reflected in the information provided in Table 1. The

341

effort for data acquisition is considerable in all three cases. In new systems or systems that

342

have received little attention thus far like minor metal systems, many information elements

343

must often be combined in order to generate the numerical value of a desired flow. Gallium

344

requires an average of more than seven information elements being necessary for

345

quantification of a flow variable, the highest among the three analyzed minor metals.

346

Comparatively few flows of the gallium system (eight out of 42) are directly quantified by

347

ready-to-apply data. Applied data in all three case studies stem partly from official and

348

scientific sources, but largely from expert estimations, industry, or assumptions (see SI2-4 for

349

details). In “well-known”, regularly monitored systems, two or three information elements

350

might be sufficient, e.g., a concentration and a flow or a price and a turnover. When facing

351

high numbers of information elements per flow, assessing all data attributes in the DCM can

352

become a daunting task. However, credibility is just as much if not more needed in emerging

353

or fast-changing systems8 than well-known systems. The DCM is most useful here, as it

354

provides a clear scheme for systematic documentation and management of a large amount of

355

data which vary greatly in quality and nature.

356

Evaluations of data attributes and information defects (IDs) are judgments made by those

357

setting up an MFA, especially for attributes regarding the semantics (attribute 202 in the

358

DCM) and complexity (attributes 211 and 212 in the DCM) of an information element. There 19 ACS Paragon Plus Environment


359

is no miracle recipe for supporting the evaluation of IDs in minor metal MFAs. Moving

360

towards more participatory forms of MFA could help increase the acceptance of ID

361

evaluations and hence the credibility of the whole MFA. We suggest here a participatory

362

process in which stakeholders of the investigated metal flow system evaluate selected IDs

363

with the explicit aim of building consensus on IDs in individual cases32.

364

As long as MFAs cannot be based upon statistically exploitable data, MFA practitioners are

365

limited to using estimations such as the IDs approach applied here. Statistical methods should,

366

of course, be applied whenever the available data are sufficient. Further research and

367

development is required to enable the hybridization of data quality evaluations (such as the

368

IDs) with statistical information. The transition from an approach based fully on data quality

369

evaluations to a statistical method will without doubt be gradual rather than abrupt, and may

370

evolve in parallel to improved data provision practices. In the case of minor metals, statistical

371

information is within reach for some material system stages, for instance for efficiencies of

372

mining and refining in Production. In that case, mines and refiners across the globe could

373

easily provide their efficiencies or a single facility could share repeated measurements of its

374

efficiency, resulting in statistical distributions useful to uncertainty analysis. Such a

375

potentially favorable knowledge situation is in stark contrast to that of Waste Management &

376

Recycling information, particularly in so far as the amounts of discarded metals are

377

concerned.

378

MFA of metals often serve the purpose of cross-case analyses, in which material

379

characteristics are compared across metals and put in contrast. Instances are studies of

380

criticality of metals and metalloids6, in-use dissipation rates33, correlations between economic

381

development and spectra of metals used34, and comparative studies of metals used in identical

382

economic activities, e.g., in clean energy technologies14. Beyond its benefits in the analysis

383

process, the reliability assessment allows communicating and comparing the reliability of 20 ACS Paragon Plus Environment

Page 20 of 27

Page 21 of 27


384

MFA results in easily accessible ways (see Table 2, Figure 4). As it is impossible to quantify

385

uncertainty in accounting MFAs on an absolute scale (“this is uncertain by plus/minus 100

386

Mg”) without sufficient statistical information and as uncertainty cannot be precisely located

387

in the system without objective information, uncertainty here is understood as a system

388

property that is represented on a systems scale (Figure 4). Such a system property can be

389

compared from metal to metal. By disaggregating the uncertainty assessment to the flow

390

level, MFA practitioners can identify where in a metal system the uncertainty is expected to

391

be the highest – the uncertainty hotspots35 – based on author evaluations of data quality.

392

Examples in this regard are provided in Schwab and Rechberger

393

documentation framework can become instrumental in setting up knowledge management

394

systems36 for the purpose of sharing data among metal MFA practitioners. We strongly

395

believe that cross-case analyses of metal characteristics would greatly benefit from a

396

transparent knowledge management system based on the DCM.

397

We emphasize that the ID approach applied in this article focuses on uncertainty resulting

398

from data quality shortcomings. Further uncertainties arising from assumptions in the model

399

used to process the MFA data (step three, as specified in the Introduction) or to mis-

400

specifications of qualitative MFA systems (step one, see SI1) are not reflected in the measures

401

illustrated in Figure 4. At an early stage of the participatory process mentioned above, MFA

402

practitioners and stakeholders should address the issue of incompleteness and errors in system

403

structure. Although such errors can be reduced by engaging experts from industries and

404

authorities in the system design process, there is a considerable degree of freedom in the

405

system design stage37 that may induce relevant uncertainties2. The degree of reconciliation is

406

a useful indicator in this regard, although the source of high values may be unclear and be a

407

result of mistakes in system design, inconsistencies in the data, or both.


19

. Ultimately, the


408

The degree of data reconciliation for MFAs with comparatively extensive and consistent

409

databases was found to be between 4 and 9% in Klinglmair, et al. 2. The degree of data

410

reconciliation for the studied minor metal MFAs ranged from 6 to 11% (Table 2), although

411

one would expect higher values because of the comparatively weak information basis. Yet, in

412

the minor metal case, much data were pre-balanced during the data collection process because

413

neighboring flows were often quantified based on similar data (a necessity induced by the

414

considerable shortage of data, cf. SI2-4). As a result, the metric is remarkably low in all

415

three systems, although the databases are observed to be comparatively weak. As more data is

416

collected on individual flows and less pre-balancing is needed, MFA practitioners should

417

expect to increase and be careful when communicating such an increase to policy-makers.

418

Confronted with several available methods for MFA reliability assessments and given limited

419

time and resources, MFA practitioners might ask which method should be used for minor

420

metal studies or for studies of other materials. In Table 3, we provide an overview of

421

advantages, besides that of providing quantitative information on the overall reliability of

422

MFAs, and disadvantages of both methods based on their general characteristics and the

423

discussion above. One should note here that they can be applied in a complementary manner.

424

For instance, the degree of data reconciliation can be used to support the design of structure,

425

while data are managed using a DCM which allows determining information content and

426

system uncertainty. Ultimately, information content and system uncertainty should be used for

427

weak knowledge situations, while the degree of reconciliation is most useful for more robust

428

knowledge situations. As statistical information becomes available, MFA practitioners should

429

move to approaches based on probability distributions3.

430


Page 22 of 27

Page 23 of 27

431 432


Table 3 Advantages and disadvantages of two methods for the assessment of MFA results reliability and recommendations for their use.

Information content and system uncertainty


+ Basis for a data/knowledge management + Support in design of system structure (step system +

one)

Break-down

of

uncertainty

into

cognitively manageable components - No basis for data/knowledge management

- Resources required to fill in DCM

- Possibly difficult evaluation of specific - Low degrees of data reconciliation can

IDs

(e.g.,

attribute

“diversity”

and result from consistency between poor data

“semantics”)

and poor system design by chance or

- No support in design of system structure because poor data bias system design (step one)

- Pre-balancing leads to a low degree of data reconciliation - No identification of uncertainty source

Recommended for situations with weak Recommended for situations with more information basis, unclear data quality and complete databases many data gaps 433 434

Concluding remarks

435

As minor metals become increasingly important in our daily lives, so does the ability to

436

provide reliable information on their amounts mined, used, dissipated, and recycled. We

437

demonstrated the feasibility of two reliability assessment methods in minor metal MFAs in

438

the absence of statistically exploitable data. The DCM approach and that of data

439

reconciliation provide overall reliability measurements that allow comparisons across time,

440

space, and metals for more effective policy and management. Based on our case applications, 23 ACS Paragon Plus Environment


441

we provided recommendations such as a participatory process for system design and IDs

442

evaluation. We highlighted the advantages, disadvantages, and complementarities that MFA

443

practitioners should expect from these procedures in the field of metal MFAs. Additional case

444

studies will facilitate comparisons of MFA systems, help map learning processes on MFAs

445

over time, and enable learning about the benefits, limitations, and pitfalls of the applied

446

procedures and other, similar approaches38, 39. Metal MFA practice still dominantly relies on

447

probability-based approaches despite repeated calls for more adequate ways to appraise

448

uncertainty. We recommend that the evaluation of reliability becomes a more integral part of

449

metal MFA practice, and that new approaches which are specific to the nature of available

450

information are proposed, tested, discussed, and refined. Finding consensus and establishing

451

best practices is an endeavor with the potential to increase the impact of MFA in decision

452

making in management and policy contexts.

453

Acknowledgements

454

The authors wish to thank the U.S. Geological Survey for support of this research. We

455

particularly thank Brian Jaskula and Desiree Polyak for valuable inputs. We acknowledge the

456

two anonymous reviewers for their insightful comments and suggestions.

457 458

Supporting information

459

SI1: Life cycle stages of rhenium, gallium, and germanium in the US economy in 2012 (PDF

460

format)

461

SI2: Data Characterization Matrix of the rhenium case study (Microsoft Excel format)

462

SI3: Data Characterization Matrix of the gallium case study (Microsoft Excel format)

463

SI4: Data Characterization Matrix of the germanium case study (Microsoft Excel format) 24 ACS Paragon Plus Environment

Page 24 of 27

Page 25 of 27


464

References

465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512

1. Lazarevic, D.; Buclet, N.; Brandt, N., The application of life cycle thinking in the context of European waste policy. J. Clean. Prod. 2012, 29-30, 199-207. 2. Klinglmair, M.; Zoboli, O.; Laner, D.; Rechberger, H.; Astrup, T. F.; Scheutz, C., The effect of data structure and model choices on MFA results: A comparison of phosphorus balances for Denmark and Austria. Resour. Conserv. Recy. 2016, 109, 166-175. 3. Laner, D.; Rechberger, H.; Astrup, T., Systematic Evaluation of Uncertainty in Material Flow Analysis. J. Ind. Ecol. 2014, 18, (6), 859-870. 4. Cencic, O., Treatment of Data Uncertainties in MFA. In Handbook of Material Flow Analysis: For Environmental, Resource, and Waste Engineers, Second Edition, Brunner, P. H.; Rechberger, H., Eds. CRC Press: Boca Raton, 2016. 5. European Commission, Report on critical raw materials for the EU. Report of the Adhoc Working Group on defining critical raw materials. European Commission's DG Enterprise and Industry: Brussels, 2014. 6. Graedel, T. E.; Harper, E. M.; Nassar, N. T.; Nuss, P.; Reck, B. K., Criticality of metals and metalloids. Proc. Natl. Acad. Sci. 2015, 112, (14), 4257-4262. 7. Graedel, T. E.; Allwood, J.; Birat, J.-P.; Reck, B. K.; Sibley, S. F.; Sionnemann, G.; Buchert, M.; Hagelüken, C., Recycling rates of metals - A status report, a report of the working group on the global metal flows to the International Resource Panel. United Nations Environment Programme (UNEP): Paris, 2011. 8. Nassar, N. T.; Graedel, T. E.; Harper, E. M., By-product metals are technologically essential but have problematic supply. Sci. Adv. 2015, 1, (3), 1-10. 9. Mudd, G. M.; Weng, Z.; Jowitt, S. M.; Turnbull, I. D.; Graedel, T. E., Quantifying the recoverable resources of by-product metals: The case of cobalt. Ore Geology Reviews 2013, 55, 87-98. 10. Stamp, A.; Lang, D. J.; Wäger, P. A., Environmental impacts of a transition toward emobility: the present and future role of lithium carbonate production. J. Clean. Prod. 2012, 23, (1), 104-112. 11. Wäger, P. A.; Lang, D. J.; Wittmer, D.; Bleischwitz, R.; Hagelüken, C., Towards a more sustainable use of scarce metals. A review of intervention options along the metals life cycle. GAIA 2012, 21, (4), 300-309. 12. Lovik, A. N.; Restrepo, E.; Muller, D. B., The Global Anthropogenic Gallium System: Determinants of Demand, Supply and Efficiency Improvements. Environ. Sci. Technol. 2015, 49, (9), 5704-5712. 13. Duclos, S. J.; Otto, J. P.; Konitzer, D. G., Design in an era of constrained resources. Mech. Eng. 2010, 132, (9), 36-40. 14. Licht, C.; Peiró, L. T.; Villalba, G., Global Substance Flow Analysis of Gallium, Germanium, and Indium: Quantification of Extraction, Uses, and Dissipative Losses within their Anthropogenic Cycles. J. Ind. Ecol. 2015, 19, (5), 890-903. 15. Løvik, A. N.; Restrepo, E.; Müller, D. B., Byproduct Metal Availability Constrained by Dynamics of Carrier Metal Cycle: The Gallium–Aluminum Example. Environ. Sci. Technol. 2016, 50, (16), 8453-8461. 16. Gunn, G., Critical metals handbook. John Wiley & Sons: Chichester (UK), 2013. 17. Navarrete-Gutiérrez, T.; Rugani, B.; Pigné, Y.; Marvuglia, A.; Benetto, E., On the Complexity of Life Cycle Inventory Networks: Role of Life Cycle Processes with Network Analysis. J. Ind. Ecol. 2016, 20, (5), 1094-1107. 18. Layton, A.; Bras, B.; Weissburg, M., Industrial Ecosystems and Food Webs: An Expansion and Update of Existing Data for Eco-Industrial Parks and Understanding the Ecological Food Webs They Wish to Mimic. J. Ind. Ecol. 2016, 20, (1), 85-98. 25 ACS Paragon Plus Environment


513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562

19. Schwab, O.; Rechberger, H., Information Content, Complexity and Uncertainty in Material Flow Analysis. J. Ind. Ecol. 2017. 20. Glöser, S.; Soulier, M.; Tercero Espinoza, L. A., Dynamic Analysis of Global Copper Flows. Global Stocks, Postconsumer Material Flows, Recycling Indicators, and Uncertainty Evaluation. Environ. Sci. Technol. 2013, 47, (12), 6564-6572. 21. Rechberger, H.; Cencic, O.; Fruehwirth, R., Uncertainty in Material Flow Analysis. J. Ind. Ecol. 2014, 18, (2), 159-160. 22. Ravetz, J. R., Scientific Knowledge and Its Social Problems. Transaction Publishers: 1973. 23. Funtowicz, S. O.; Ravetz, J. R., Uncertainty and Quality in Science for Policy. Springer: 1990. 24. Schwab, O.; Zoboli, O.; Rechberger, H., A Data Characterization Framework for Material Flow Analysis. J. Ind. Ecol. 2017, 21, (1), 16-25. 25. Schwab, O.; Laner, D.; Rechberger, H., Quantitative evaluation of data quality in regional Material Flow Analysis. J. Ind. Ecol. 2016. 26. Zoboli, O.; Zessner, M.; Rechberger, H., Added Value of Time Series in MFA: The Austrian Phosphorus Budget from 1990 to 2011. J. Ind. Ecol. 2016, 20, (6), 1334–1348. 27. Cencic, O.; Rechberger, H., Material flow analysis with software STAN. Journal of Environmental Engineering and Management 2008, 18, (1), 5. 28. Graedel, T. E.; Barr, R.; Chandler, C.; Chase, T.; Choi, J.; Christoffersen, L.; Friedlander, E.; Henly, C.; Jun, C.; Nassar, N. T.; Schechner, D.; Warren, S.; Yang, M.-y.; Zhu, C., Methodology of metal criticality determination. Environ. Sci. Technol. 2012, 46, (2), 1063-1070. 29. Hekkert, M. P.; Suurs, R. A.; Negro, S. O.; Kuhlmann, S.; Smits, R., Functions of innovation systems: A new approach for analysing technological change. Technol. Forecast. Soc. Change 2007, 74, (4), 413-432. 30. Markard, J.; Truffer, B., Technological innovation systems and the multi-level perspective: Towards an integrated framework. Res. Pol. 2008, 37, (4), 596-615. 31. Binz, C.; Truffer, B., Technological Innovation systems in multiscalar space: Analyzing an emerging water recycling industry with social network analysis. Geogr. Helv. 2011, 66, (4), 254-260. 32. Krütli, P.; Stauffacher, M.; Flüeler, T.; Scholz, R. W., Functional‐dynamic public participation in technological decision‐making: site selection processes of nuclear waste repositories. J. Risk Res. 2010, 13, (7), 861-875. 33. Ciacci, L.; Reck, B. K.; Nassar, N. T.; Graedel, T. E., Lost by Design. Environ. Sci. Technol. 2015, 49, (16), 9443-51. 34. Graedel, T. E.; Cao, J., Metal spectra as indicators of development. Proc. Natl. Acad. Sci. 2010, 107, (49), 20905-20910. 35. Prestele, R.; Alexander, P.; Rounsevell, M. D. A.; Arneth, A.; Calvin, K.; Doelman, J.; Eitelberg, D. A.; Engström, K.; Fujimori, S.; Hasegawa, T.; Havlik, P.; Humpenöder, F.; Jain, A. K.; Krisztin, T.; Kyle, P.; Meiyappan, P.; Popp, A.; Sands, R. D.; Schaldach, R.; Schüngel, J.; Stehfest, E.; Tabeau, A.; Van Meijl, H.; Van Vliet, J.; Verburg, P. H., Hotspots of uncertainty in land-use and land-cover change projections: a global-scale model comparison. Global Change Biol. 2016, 22, (12), 3967-3983. 36. Alavi, M.; Leidner, D. E., Review: Knowledge Management and Knowledge Management Systems: Conceptual Foundations and Research Issues. MIS Quarterly 2001, 25, (1), 107-136. 37. Heijungs, R., Topological network theory and its application to LCA and IOA and related industrial ecology tools: principles and promise. J. Environ. Account. Manage. 2015, 3, (2), 151-167. 26 ACS Paragon Plus Environment

Page 26 of 27

Page 27 of 27

563 564 565 566 567


38. Hedbrant, J.; Sörme, L., Data Vagueness and Uncertainties in Urban Heavy-Metal Data Collection. Water Air Soil Pollut. Focus 2001, 1, (3-4), 43-53. 39. Laner, D.; Feketitsch, J.; Rechberger, H.; Fellner, J., A novel approach to characterize data uncertainty in MFA and its applications to plastic flows in Austria. J. Ind. Ecol. 2016, 20, (5), 1050-1063.

568


Assessing the Reliability of Material Flow Analysis Results: The Cases

Recommend Documents