Subscriber access provided by Columbia University Libraries
Article
IDENTIFICATION OF LUMINESCENT MARKERS FOR GUNSHOT RESIDUES: FLUORESCENCE, RAMAN SPECTROSCOPY AND CHEMOMETRICS Caroline Ribeiro Carneiro, Carolina Santos-Silva, Marcela Albino Carvalho, Maria Fernanda Pimentel, Marcio Talhavini, and Ingrid Tavora Weber Anal. Chem., Just Accepted Manuscript • DOI: 10.1021/acs.analchem.9b03079 • Publication Date (Web): 01 Sep 2019 Downloaded from pubs.acs.org on September 1, 2019
Just Accepted “Just Accepted” manuscripts have been peer-reviewed and accepted for publication. They are posted online prior to technical editing, formatting for publication and author proofing. The American Chemical Society provides “Just Accepted” as a service to the research community to expedite the dissemination of scientific material as soon as possible after acceptance. “Just Accepted” manuscripts appear in full in PDF format accompanied by an HTML abstract. “Just Accepted” manuscripts have been fully peer reviewed, but should not be considered the official version of record. They are citable by the Digital Object Identifier (DOI®). “Just Accepted” is an optional service offered to authors. Therefore, the “Just Accepted” Web site may not include all articles that will be published in the journal. After a manuscript is technically edited and formatted, it will be removed from the “Just Accepted” Web site and published as an ASAP article. Note that technical editing may introduce minor changes to the manuscript text and/or graphics which could affect content, and all legal disclaimers and ethical guidelines that apply to the journal pertain. ACS cannot be held responsible for errors or consequences arising from the use of information contained in these “Just Accepted” manuscripts.
is published by the American Chemical Society. 1155 Sixteenth Street N.W., Washington, DC 20036 Published by American Chemical Society. Copyright © American Chemical Society. However, no copyright claim is made to original U.S. Government works, or works produced by employees of any Commonwealth realm Crown government in the course of their duties.
Page 1 of 10 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60
Analytical Chemistry
IDENTIFICATION OF LUMINESCENT MARKERS FOR GUNSHOT RESIDUES: FLUORESCENCE, RAMAN SPECTROSCOPY AND CHEMOMETRICS. Caroline R. Carneiroa, Carolina S. Silvab, Marcela Albino de Carvalhob, Maria Fernanda Pimentelb, Márcio Talhavinic, Ingrid T. Webera*. a LIMA,
Chemistry Institute, University of Brasília – UNB, P.O. Box 04478, 70904-970 Brasília, Brazil Department of Chemical Engineering, Federal University of Pernambuco – UFPE, Av. Prof. Moraes Rego, 1235, Cidade Universitária, 50740-540 Recife, Brazil c National Institute of Criminalistics, Brazilian Federal Police, SAIS Quadra 07 Lote 23, 70610-200 Brasília, DF, Brazil. *
[email protected], phone: +55 61 31073801 b
ABSTRACT: Gunshot residue (GSR) is an evidence of major importance in firearm-related crimes. The recent introduction of non-toxic ammunition has made impossible the characterization of GSR particles by the current methods employed by forensic experts. To overcome this drawback, the introduction of luminescent markers was proposed, allowing on site visual detection of luminescent gunshot residue (LGSR) at the crime scene. Three different luminescent markers coordinated with europium, for specific and selective encoding of ammunition have been proposed. To promote a variety of versatile tools for GSR analysis, spectroscopic techniques combined with chemometric methods can be applied to achieve a reliable, fast and non-destructive means to identify LGSR and discriminate among the different markers. Luminescence (emission and excitation), normal and resonance Raman spectroscopies associated with Principal Component Analysis (PCA) and Partial Least Squares - Discriminant Analysis (PLS-DA) were evaluated. The classification model using the complementary information of emission and excitation spectra, a.k.a. data-fusion, provided a 100% correct classification for all markers. A comprehensive study has been developed to show that the insertion of luminescent markers enables not only the easy localization of GSR residues, but also the possibility of ammunition encoding through the use of multivariate classification methods. INTRODUCTION in scenarios that may not be related to criminal events, Gunshot Residue (GSR) is an extremely important type of thus generating a number of false positives. Examples of evidence in firearm-related crimes1. It can be used to these are car brake pad particles18, automotive hybrid determine whether a suspect has fired a weapon, to link airbags19, fireworks20, and particles from the welding suspects and victims to a crime scene, to help to process21. Therefore, the existence of many false-positives can still hamper the unequivocal identification of GSR from differentiate suicide from homicide, to determine the distance of the shot, recognize bullet entry wounds from Non-Toxic Ammunition based on its organic compounds, firearm damage, among others2-6. GSR is traditionally thus it is still an important piece of evidence that still lacks characterized by the simultaneous presence of lead (Pb), of validated methodologies for its identification. antimony (Sb) and barium (Ba) in spheroid shape In order to circumvent this problem, markers have been particles7. Currently, the most reliable GSR incorporated into ammunition. Special marked characterization technique used is scanning electron ammunition has been already used in some European microscopy coupled with Energy Dispersive X-Ray countries. For instance, one type of ammunition produced Spectroscopy (SEM-EDS). This technique allows a joint in Switzerland contains Gd as the marking element and a analysis of both the morphology and the individual German ammunition uses Ga in gunpowder22. Our research chemical composition of the particles8,9. group has suggested the incorporation of luminescent In the recent years, studies have shown that the high markers based on Metal-Organic Frameworks (MOFs) and toxicity of the heavy metals present in regular ammunition lanthanide ions in ammunition23-25. Luminescent markers 10 constitute a risk to frequent shooters . This has led to the outperform chemical markers because, in addition to development of Non-Toxic Ammunition, also known as providing a unique composition to the GSR, they permit lead-free ammunition. What was considered a visual detection of Luminescent Gunshot Residue (LGSR) in breakthrough in an occupational safety and health context, loco, facilitating the identification of GSR particles with a imposed a drawback for ballistics, since the available portable ultraviolet lamp. The LGSR can be easily techniques for GSR identification could no longer be visualized on the hands, weapons, clothing, as well as at the shooting area and on the victim, enabling employed11,12. Given this, the scientific community has been dedicating establishment of a link among them. Previous studies have efforts to the analysis of Organic Gunshot Residues (OGSR) confirmed the efficiency of the luminescent marker in in order to characterize GSR from Non-Toxic Ammunition forensic routines25, which have shown satisfactory residues13. Techniques such as Raman Spectroscopy14,15, performance in important questions such as: kinetic Infrared16, and liquid chromatography coupled to mass energy of the projectile and failure rate, LGSR's length of spectrometry17 have been explored. Some studies have stay, resistance to hand washing, LGSR transfer through suggested the possibility of finding particles similar to GSR object manipulation by the shooter, LGSR detection in
ACS Paragon Plus Environment
Analytical Chemistry 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60
unconventional sites and dispersal of the LGSR in a simulated crime scene23. The aggregation of lanthanides as the metallic center in MOFs is a strategy to generate luminescent sensors. Among the lanthanides, the europium ion (III) is a particularly attractive luminophore due its spectrally narrow and well defined emission in the red region26. In addition, the narrow spectral lines and as well the relative intensities of the transitions in the luminescence spectra of Eu3+ can be used to probe the local environment of the ion. The spectroscopic data provide information about the point group symmetry of the Eu3+ site and information about the coordination polyhedron27. By changing the lanthanide ion, it is possible to differentiate markers by the color of the emitted light28. If the same ion is fixed, it is possible to differentiate the marker by analyzing the organic ligands. Our group has applied Near Infrared Hyperspectral Images combined to chemometric techniques to distinguish three luminescent markers based on changes of organic ligands. As result, 72.2% of the samples were successfully identified29, but misclassification problems were encountered due the amount of collected material or because one marker was not properly characterized. In this context, other analytical methods and spectroscopic techniques have been employed to provide reliable results and unequivocally identification of LGSR, allowing ammunition encoding and further traceability. However, spectroscopic data generates a large amount of data, making the specialist's task tedious, time-consuming, and manual examination can yield false positive results30. Multivariate Analysis techniques (a.k.a. Chemometrics) can be employed to develop methods for the analysis of large and complex chemical data sets. Classification techniques such as PLS-DA31-33 can be employed for ammunition traceability, which can be used to establish gun-shootervictim-local correspondence in crime scene, making its application crucial to forensic sciences. The aim of this work is to evaluate the potential of Luminescence and Raman spectroscopies associated to chemometric techniques to propose a new analytical methodology for ammunition encoding. To do this, three different MOF-based luminescent markers were produced, inserted in Non-Toxic Ammunition firearms ammunition and the LGSR were analyzed by the abovementioned analytical techniques. Different approaches for classification have been employed to discriminate the luminescent markers according its molecular structures, which contain the same emitter but different organic connectors. CHEMOMETRICS Principal Component Analysis Principal Component Analysis (PCA) is one of the most important and well-known methods in chemometrics that aims to represent a particular set of data in terms of maximum variance and, consequently, reduce the number of dimensions of the dataset in order to facilitate their interpretation34. It is an exploratory analysis method, in which a multivariate interpretation of a complex dataset is enabled without the need for initial information regarding the nature of the samples. PCA constructs a new vector space in which new variables, called Principal Components
Page 2 of 10
(PC), which are not correlated, are developed from a linear combination of the original variables of a given data matrix35. Partial Least Squares – Discriminant Analysis Among the classification methods, Partial Least Squares – Discriminant Analysis (PLS-DA) stands out as a supervised method which correlates spectroscopic information with one or more chemical or physical properties of interest, which contains the classes to which each sample of the training set belongs. The PLS-DA makes use of unique binary code and during the calibration process is trained to calculate the so-called "adhesion" values for each class. Since this value is above a specific threshold for prediction the sample is therefore assigned to a single class36. While PCA builds a model based on maximum variance of the spectral information, PLS-DA is able to model the dataset based in the maximum correlation between spectra and classes. DATA-FUSION Data fusion consist of a multiblock modeling technique that enables the analyst to enhance the chemical information from a set of samples by using complementary inputs. This makes it possible to combine different analytical techniques so that the information sought will be available in the joint data, but not in the individual blocks37. There are three main categories of fused data: (i) low-level, (ii) mid-level or hierarchical data fusion; and (iii) high-level or decision level data fusion38. In low-level data fusion, spectral profiles are concatenated to provide information of different nature. In the case of mid-level, a feature extraction step or dimensionality reduction is performed and the resulting information is fused. Finally, in high-level cases, a supervised model (classification or regression) is built and the combined results will provide a final response. This emerging technique of analysis can provide valuable information on complex data sets. It has been employed in different fields, and to the best of our knowledge, no forensic application has been reported up to now. Further information on the abovementioned chemometrics techniques can be found elsewhere31,34,37-41. MATERIALS AND METHODS Sample preparation Three MOFs with potential for GSR markers were synthesized: [Eu(DPA)(HDPA)], [Eu(BTC)] and [Eu2(BDC)3(H2O)2]n, from now one referred as Eu(DPA), Eu(BTC) and Eu(BDC), respectively. Those MOFs were chosen based on previous studies conducted by the group23-25,45. For Eu(DPA) marker, a mixture of Eu(NO3)3∙6H2O (0.35 mmol), dipicolinic acid (H2DPA) (0.7 mmol) and 4 ml of distilled water were added to a 10 mL quartz reactor. The reaction was performed at 160°C for 20 minutes. For the Eu(BTC) marker, Eu2O3 (0.175 mmol), trimesic acid (H3BTC) (0.35 mmol) and 4 mL of distilled water was mixed and placed in a 10 mL quartz reactor. The reaction was performed at 150°C for 20 minutes. And for the Eu(BDC) marker, a mixture of Eu(NO3)3∙6H2O (0.7 mmol), terephthalic acid disodium salt (Na2BDC) (0.7 mmol) and 4 mL of distilled water was placed in a 10 mL quartz reactor, under agitation. The reaction was performed at 160°C for 20 minutes. After each reaction, the materials obtained were washed with water and
ACS Paragon Plus Environment
Page 3 of 10 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60
Analytical Chemistry
acetone and dried at 100°C for 24 hours. All the samples were hydrothermally prepared, using a microwave reactor and autogenous pressure (Monowave 300 Anton Paar) with power of 400 W. Each marker synthetized was added to the gunpowder of 9 mm Non-Toxic Ammunition (Clean Range® CBC) at a ratio of 5% wt in order to produce 30 LGSR samples. The shots were carried out at the indoor shooting range of the ballistics service of the INC/DPF-DF (National Institute of Criminalistics of the Brazilian Federal Police, Brasília). Thirty shots were fired at a short distance (30 cm), towards paperboard targets covered with black cotton cloth, using Jericho 941F and Glock G17 pistols (15 of each pistol). Spectral Acquisition and MOF characterization Each synthesis was performed five times, resulting in 15 samples, 5 Eu(BTC), 5 Eu(DPA), and 5 Eu(BDC). Structure of samples were evaluated by X-ray diffraction (Rigaku/MiniFlex 300), with the following parameters: 2θ range from 5º to 50º with 0.01 of step and 1.5º/min of speed. The LGSR samples were collected from the targets individually using carbon tape stubs. Spectra were acquired of pure markers (MOFs prior the shot) and the collected LGSR (marker post shot) under the same conditions. Different spectroscopic techniques were employed: the excitation and emission acquired with a fluorometer and Raman spectra and emission spectra obtained by the resonance Raman effect. Emission and Excitation Spectra Excitation spectra of all syntheses of the three markers (15 samples) were acquired with a Fluorolog spectrofluorometer from Horiba Scientific by monitoring the emission at 613 nm. The spectral range analyzed was between 200 nm and 550 nm. The emission spectra, on the other hand, were acquired by monitoring the excitation at 293 nm and analyzing in the range of 550 to 750 nm. All emission spectra were obtained using 1.5 nm slit and integration time of 1 s. No sample preparation was needed before the analysis. The excitation spectra, in turn, were obtained using 1.0 nm slit and integration time of 0.2 s. Normal and Resonance Raman Spectra The samples of pure markers (MOFs prior the shot) and the LGSR (marker post shot) were also analyzed in a Raman spectrometer (Bruker SENTERRA) coupled with a confocal microscope. Again, no sample preparation was needed. Two lasers were employed for spectral acquisition, the 532 nm and the 785 nm. The 532 nm laser generates the resonance Raman effect due the fact that the incident photons have energy quite similar to the Eu3+ excitation (7F05D0)42,43, therefore emission spectra were obtained using this configuration. To differentiate it from the emission spectra obtained with the fluorometer, in this work this is referred to as Resonance Raman spectra, although the spectral profile corresponds, in fact, to Eu3+ emission. Both techniques were evaluated because although Raman spectroscopy is less straightforward for the analysis of luminescent samples, it is more commonly used in forensic laboratories. In addition, the 785 nm laser was employed to acquire information about the vibrational spectral profile of the samples, here referred to as normal Raman spectra. At least, 5 spectra were acquired for each sample. Error! Reference source not found. shows the
parameter employed during spectral acquisition for each laser. Chemometric treatment For all techniques, acquired spectra were averaged, resulting in four different datasets according the analytical technique employed (emission, excitation, normal Raman and resonance Raman) containing 45 samples. Thirty samples were used as the training set (15 of pure markers, 5 from each, and another 15 samples of LGSR collected after shooting session performed with the Jericho 941F pistol). For the test set, the other 15 samples of LGSR were employed, those produced with the Glock G17 pistol. Spectral preprocessing techniques such as normalization, standard normal variate, baseline correction (automatic weighted least squares) and mean centering were performed on the datasets (fluorescence and Raman). Those techniques are important in removing from data unwanted variation related to noise, intensity differences, fluorescence and other spectral artifacts. PCA was carried out for samples of each marker to identify and eliminate outliers. Afterwards, PCA was performed for all remaining samples to visualize variations within the datasets. TABLE 1. Parameters employed during spectral acquisition using the 532 nm and 785 nm lasers.
PARAMETER Confocal Aperture (pinhole) Laser power Magnification Integration time Coadditions Range (cm-1)
532 nm
785 nm
50 µm
50 µm
0.2 mW 50x 0.1s 1 699 – 4400
10 mW 50x 5s 2 559 – 1809
Since the main objective of the study was to encode ammunition, classification techniques were employed. In a future use, types and compositions of all markers incorporated in ammunition will be known and controlled, which justifies the use of discriminant analysis techniques. Therefore, Partial Least Squares – Discriminant Analysis (PLS-DA) models were built to classify LGSR according the type of marker. Mid-level data fusion was employed using PCA scores to improve classification results. All chemometric treatments were performed using the PLS_Toolbox (Eigenvector Research Inc., USA), in MATLAB environment. RESULTS AND DISCUSSION MOFs’ X Ray Diffraction Characterization Regarding the structural characterization of the markers, the X-ray powder diffraction patterns were compared with the diffraction patterns presented in the literature through Crystallographic Information Files (CIF). The molecular structure from the ligand employed and the diffractograms are shown in the supporting information (Figures S1-S4). The Eu(DPA) marker diffraction, Figure S1, showed peaks inconsistent with the database pattern, which probably indicates that there was a mixture of phases in the synthesized MOF. The presence of secondary phases can be clearly seen in the peaks that appear between 7 and 13º. In addition, there is a large fluctuation in the profile of the diffractograms obtained by the 5 different syntheses, showing that this marker exhibits a structural repeatability
ACS Paragon Plus Environment
Analytical Chemistry 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60
problem. Despite this, the optical properties of the marker were maintained and its function as marker was satisfied. On the other hand, the diffractogram of the Eu(BTC) and Eu(BDC) markers, Figures S3 and S4, respectively, were consistent with the standard. There are small discrepancies between the standard used for comparison and the synthesized samples, for example, the relative peak intensity between 25 and 30° for the Eu(BTC) samples and the 30° peak for the Eu(BDC) samples. Variations in relative intensity, however, can be attributed to preferential orientation. Eu(BDC), in particular, exhibited the best repeatability between syntheses performed. Spectral Analysis Figure 1 shows the Emission and Excitation spectra acquired for all markers and LGSR samples collected. Figure 1A and B shows the effect of preprocessing techniques for the excitation profiles. standard normal variate, Savitzky-Golay smoothing filter (2nd order polynomial and 11-point window width) and baseline correction were performed to attenuate spectral artifacts. In all excitation spectra (Figure 1B) it is possible to observe characteristic transitions of Eu3+ as well as a large band (between 250 nm and 350 nm). This band corresponds to the transition π π * (phosphorescence of the ligand) and is responsible for the antenna effect. Although noisy spectral profiles are noticed, the mentioned band is highly selective according to the organic ligands. The emission spectra (Figure 1D), on the other hand, was preprocessed with normalization by the maximum and baseline correction and it enabled observation of the characteristic transitions around 605 to 620 nm related to Eu3+, particularly the transition 5D0 7FJ, as J = 0 - 4. The most important peaks can be noticed around 570 nm and 580-600 nm related to 7F05D0 and 7F05D1 transitions, respectively. 10 5 (A) Excitation Raw Spectra 8
6
(B) Excitation Preprocessed Spectra
5 4
6
3 4
2 1
2
0 0 200
300
400
500
-1
200
300
wavelength(nm) 7
400
500
wavelength(nm)
10 4 (C) Emission Raw Spectra
(D) Emission Preprocessed Spectra 1
6 5
0.8
4
0.6
3
which this ion is found. These spectroscopic data provide information about the symmetry in which Eu3+ is inserted. The 5D07F0 transition (around 575 nm) in the Eu(DPA) and Eu(BTC) markers spectra indicates that Eu3+ occupies a site with Cnv, Cn or Cs symmetry27. This information agrees with the symmetry observed in these two MOFs. The Eu(BTC) has C2 symmetry and the Eu(DPA) has C2v symmetry44,45. This transition may also assist in determining the presence or absence of more than one non-equivalent europium site. As the Eu(BTC) spectra show only one peak in the region equivalent to this transition, it is estimated that this marker has a single equivalent Eu3+ site. The transition 5D07F1 (around 590 nm, Figure 1D) indicates the presence of hexagonal, tetragonal, and trigonal crystalline fields. For the three markers, the 5D0 → 7F (around 615 nm) transition appears as the most 2 intense, what can be explained by the high polarizability of the chelating agent. In the Eu(BTC) marker the transitions 5D 7F and 5D → 7F exhibit similar relative intensities, 0 1 0 2 indicating that the europium ion is inserted in an environment of high symmetry. The 5D0 → 7F3 (around 650 nm) transition appears with low intensity in the spectra of DPA and BTC because it is forbidden according to the JuddOfelt theory27. Its absence is related to the low spin-orbit coupling. When comparing the excitation and emission spectral profiles from the pure MOFs and the LGSR samples, no apparent significative differences are observed, except the fact that excitation profiles from LGSR show low S/N ratio when compared to the pure markers and some baseline problems for Eu(BTC) samples (Figure S5, in supporting Information). Regarding Raman analysis, Figure 2 shows the spectra obtained in both normal (Figure 2A and B) and resonance (Figure 2C and D) Raman. Both spectral profiles were preprocessed with normalization and automatic weighted least squares baseline correction, although some baseline artifacts due the high fluorescence influence were not completely eliminated by the preprocessing. Regarding the normal Raman profiles (Figure S6), it is possible to notice high variability among the spectra. Moreover, two bands around 1300 and 1600 cm-1 are observed only for LGSR profiles because they are related to the carbon tape material of the stubs employed to collect the GSR. Figure S5A from supporting information shows more details on differences and similarities of spectral profiles from pure markers and LGSR for normal Raman.
0.4
2 0.2
1
0
0 550
600
650
700
750
wavelength(nm)
800
550
600
650
700
750
800
Page 4 of 10
850
wavelength(nm)
Figure 1 (A) Raw and (B) preprocessed Excitation spectra for all markers and LGSR samples; (C) Raw and (D) preprocessed Emission spectra for all markers and LGSR samples. Red lines represent Eu(BDC) samples, green lines for Eu(BTC) and blue lines for Eu(DPA).
The spectral profile and the relative intensity of the transitions observed in the Eu3+ emission spectrum provide information about the chemical environment in
ACS Paragon Plus Environment
Page 5 of 10 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60
Analytical Chemistry
Figure 2 (A) Raw and (B) preprocessed normal Raman (785 nm) spectra for all markers and LGSR samples; (C) Raw and (D) preprocessed resonance Raman (532 nm) spectra for all markers and LGSR samples. Red lines represent Eu(BDC) samples, green lines for Eu(BTC) and blue lines for Eu(DPA).
The resonance Raman spectra (Figure 2C and 2D) provided the similar information as the emission spectra obtained in the fluorimeter. In this case, the same transitions, as expected, are observed. However, due to Raman’s point-by-point characteristics, small particles are analyzed one at a time. Therefore, more variation is observed since different particles might show different profiles due the structural changes that materials undergoes upon firing as well as the repeatability problem previously mentioned and demonstrated by the diffractograms (specially for Eu(DPA) samples). Resonance Raman does not show substrate influence due the high increase on signal intensity obtained by the resonance phenomenon. Details of spectral profiles of resonance Raman from pure markers and LGSR can be seen in Figure S6B in the supporting information. Principal Component Analysis Principal Component Analysis (PCA) was performed for preprocessed, mean-centered datasets. Regarding excitation spectra, the first two principal components explain approximately 95% of data variance. Excitation and emission spectra show interesting features in the scores scatter plots (Figure 3), where it is possible to visualize a clear differentiation of Eu(BDC) samples for excitation profiles, while the emission profiles show the same behavior for Eu(BTC) samples.
Figure 3 PC1 and 2 from PCA analysis of Excitation (A and B) and Emission spectra (C and D). Scores scatter plot (A and C) and loading plots (B and D).
The score scatter plot (Figure 3A) shows clear differences between the three markers of PC1 and PC2: the 1st PC explains the differences between the Eu(BDC) marker and the others, while the 2nd PC highlights the differences between Eu(BTC) and Eu(DPA). These are related to the region between 300 and 350 nm corroborating with the spectral profiles (Figure 1B), in which the π π * transition is shifted in Eu(BDC) when compared to the others. In any case, the 2nd PC shows that the differences between Eu(BTC) and Eu(DPA) are also related to a shift in band around 250 nm and 350 nm and an intense peak at 395 nm, which is more intense for Eu(DPA) samples due the transition 5L6 7F0. PCA results of emission spectra are observed in Figure 3C and 3D. The 2 first PCs explains 96% of data variance. Data maximum variance is related to the differences between Eu(BTC) samples the other two markers, due the relative similar intensity of the transitions 7F05D1 (around 590 nm) and 7F05D2 (around 615 nm) which does not occur for the Eu(DPA) and Eu(BDC). The latter two are distinguished one from the other due the 7F05D4 transition (around 700 nm). The 7F05D2 (around 615 nm) has a great influence on this differentiation, possibly due the hypersensitive transition being greatly influenced both by the symmetry of Eu3+ as well as the ligands that surround it. The LGSRs samples of the Eu(BTC) and Eu(BDC) markers are associated with the pure marker samples, suggesting the possibility, by means of classification techniques, to predict the MOF used in marking the ammunition. For the Eu(DPA) marker, there is the possibility of structural modifications at the moment of the shot that lead to small modifications in the spectral profile of this marker. This makes a difference between the samples of the pure markers and the LGSR, as can be observed. In addition, possible confusion between the LGSR-Eu(DPA) samples and the LGSR-Eu(BDC) samples is expected, probably in function of the similar emission spectral profile of these two markers. PCA results for 3PCs are shown in Figure S7 (supporting information).
ACS Paragon Plus Environment
Analytical Chemistry 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60
PCA was also performed for both Raman datasets (Error! Reference source not found.). In the case of normal Raman spectra, since the substrate can influence the results and cause an undesired variability between markers and LGSR samples, the regions related to its contributions were removed. This can be done, because the process of collecting LGSR samples involves the use of a substrate that is not employed when analyzing the pure marker. Error! Reference source not found.A and B show the results of PCA applied to normal Raman spectra. Even after removing variables related to the substrate, it is still possible to observe differences between the pure marker and LGSR samples, because other components of GSR (such as OGSR and dirt) can also provide a Raman signal unlike the excitation and emission spectra.
Page 6 of 10
The two first Principal Components (PCs) are able to show the differences between samples containing the different markers. The highest variability of the data, represented by PC1 and explaining almost 54% of data variability, is related to the difference between samples containing Eu(BDC) and the other two markers. This can be explained by the spectral regions at 870, 1070 and 1454 cm-1, related to samples containing Eu(BDC) and 1000 and 1470 cm-1. In contrast, PC2 explains 29% of data variability and shows the difference between samples containing Eu(BTC) and Eu(DPA) with similar spectral regions involved. The third PC is also informative (see Figure S8A and B at supporting information) and, even though relative confusion between samples is observed in the lower score values for PC1, PC2 and PC3, a clear differentiation is observed.
Figure 4 PC1 and 2 from PCA analysis of normal Raman (A and B) and resonance Raman (C and D). Scores scatter plot (A and C) and loading plots (B and D). TABLE 2 PLS-DA results for all datasets. (LV = Latent Variables; TPR = True Positive Rate; TNR = True Negative Rate).
Calibration TPR TNR BDC 1.00 1.00 Emission 3 BTC 1.00 1.00 DPA 1.00 1.00 BDC 1.00 1.00 Excitation 6 BTC 1.00 1.00 DPA 1.00 1.00 BDC 0.90 0.95 N. Raman 3 BTC 1.00 0.95 DPA 0.90 1.00 BDC 1.00 0.94 R. Raman 3 BTC 1.00 1.00 DPA 0.88 1.00 BDC 1.00 1.00 Data Fusion 2 BTC 1.00 1.00 DPA 1.00 1.00 Regarding resonance Raman spectra, PCA was more selective. Fewer differences between pure markers and LGSR are observed, due the fact that signal arises only from the Eu3+ ion. Therefore, OGSR and substrate bands do not interfere in the LGSR spectra. It is important to mention that, although the emission spectra are not a characterization of the ligand per se, emission spectra can Technique
LV
Ligand
Cross-validation Test TPR TNR TPR TNR 1.00 1.00 0.80 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 0.90 1.00 0.95 1.00 1.00 0.60 0.95 1.00 1.00 0.90 0.85 1.00 1.00 0.80 0.95 1.00 1.00 0.90 0.95 0.80 1.00 0.90 0.90 1.00 0.90 1.00 0.94 1.00 1.00 1.00 1.00 1.00 1.00 0.88 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 be highly selective, since the lanthanide environment will change according the ligand. Resonance Raman profiles provide, as expected, very similar results when compared with the emission profiles obtained with the fluorometer (Figure 3A). In both cases, samples containing the Eu(DPA) marker show high dispersion, once more due their structural changes upon firing and synthesis repeatability
ACS Paragon Plus Environment
Page 7 of 10 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60
Analytical Chemistry
problem. In fact, the dispersion of Eu(DPA) samples is problematic for all analytical techniques, showing confusion with Eu(BTC) samples in Excitation and normal Raman data; and a subtle confusion with Eu(BDC) for Emission and Resonance Raman data. PLS-DA The PCA results made it possible to observe not only similarities and differences between the markers, but also the repeatability of synthesis. This is valuable information that indicates the possibility of differentiating the three markers as well as LGSR samples. Therefore, a classification technique was employed for ammunition encoding purposes. For future application, controlled tagging of ammunition using luminescent markers can be developed and therefore Partial Least Squares – Discriminant Analysis (PLS-DA) could be employed for LGSR classification. As previously mentioned, the training set for all datasets was built with 15 samples of the pure markers and 15 samples of LGSR collected after shooting, 5 from each marker. An external test set was built with another 15 samples of LGSR from other shooting session. Random subset cross-validation was performed. Error! Reference source not found. shows the resume the results of the PLS-DA models. Ratios of True Positive (TPR) and True Negative (TNR) results are shown for each modeling step: calibration, validation and test. In general, it is possible to see that high values for TPR and TNR are obtained, with similar performances. However, regarding the test set, excitation and resonance Raman performed better with the model, despite some misclassification problems during validation. Resonance Raman showed no problems in test set, although during validation, a sample containing Eu(DPA) was classified as Eu(BDC). This corroborates the initial confusion showed in the PCA score plots for both emission datasets (acquired with the fluorometer and resonance Raman), in Figures 4C and 3C. Regarding Excitation data, 5 samples were misclassified; specifically, 3 samples containing Eu(BTC) markers were classified as Eu(DPA), at validation, which also corroborates the PCA results in Figure 3A. Normal Raman models have shown lower ability to differentiate the three markers, very likely because of the contributions of substrate, OGSR, dirt, etc., as previously discussed. In total, a number of samples were misclassified in all steps of modeling process. In the validation step, one sample containing Eu(DPA) was classified as Eu(BDC), other two samples from Eu(BDC) were misclassified, one as Eu(BTC) and the other as Eu(DPA); and, finally one Eu(BTC) sample was classified as Eu(DPA). This is related to the complex profile of Normal Raman spectra, which also provide information about OGSR common to the three types of LGSR samples, regardless the marker. The Variable Importance in Projection (VIP)44 is shown in Figure 5, in which the significance threshold indicates the most important variables in projection for each class modeled. Figure 5A shows the VIP scores for the excitation data, where Eu(BDC) samples are particularly different from the others, which explains the difference in VIP scores profile for Eu(BDC), especially at 300-350 nm. The similarity between VIP scores of Eu(BTC) and Eu(DPA) can be understood as a depiction of the confusion between the two classes. This also applies to the emission profiles in Figure 5B, in which Eu(BTC) is particularly different from
the other two. Not surprisingly, for both emission and resonance Raman data, equivalent variables show significant importance in projection. Particularly the main transitions 7F05D1 and 7F05D2, which appear around 580 nm and 610 nm in emission data, respectively, and 1800-1900 cm-1 and 2500 cm-1 for resonance Raman. The highest VIP scores in both cases are related to the europium luminescence, which, as mentioned is highly sensitive to its chemical environment defined by the ligand.
Figure 5 VIP scores of PLS-DA models for (A) excitation, (B) emission, (C) normal and (D) Resonance Raman data.
Data Fusion The most interesting feature about excitation and emission spectra is their complementary characteristics. Observing the scores scatter plot of the PCA results (Figure 3) and the PLS-DA results (Error! Reference source not found.), it is possible to visualize that the misclassification problems of the two analytical techniques were not the same. Thus, a mid-level data fusion approach was employed, using the scores of the first 4 PCs of excitation and emission data after autoscaling. The PCA results of fused data are shown in Figure 6. A clear improvement is observed in PCA results, revealing a clear distinction among the luminescent markers with respect to the ligand employed. As the π-π* band responsible for distinguishing the Eu(BDC) marker in excitation spectra and emission spectra the relative intensity of transitions 5D0-7F1 and 5D0-7F2 allows an easy distinction of the Eu(BTC) marker. PLS-DA model was also performed for the fused data and the results are shown in Error! Reference source not found. and Figure 7. As it can be seen, a perfect classification was achieved using the data-fusion approach, providing an unequivocal identification of all 15 LGSR test samples, resulting in 100% accuracy of the technique. The advantage of employing fused emission and excitation information is that it is also valid for a routine application purpose, since the same equipment can be employed and more than one analytical technique used to ensure a reliable result. When compared with previous works developed by our group29, it is possible to notice advantages and drawbacks
ACS Paragon Plus Environment
Analytical Chemistry 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60
from both methodologies proposed. Whereas image analysis provides a faster result and the additional information of particle distribution, the luminescence spectra is not affected by the substrate information nor the OGSR compounds present. The latter provided a better classification result due the high selectivity of the analytical technique. Both proposed methods are nondestructive and non-invasive, therefore adequate to be part of the routine of a scientific police laboratory.
Figure 6 (A) Scores scatter plot and (B) loadings plot for PC1, PC2 and PC3 of mid-level fused data.
Page 8 of 10
approach was performed to improve the models, providing 100% correct classification for all samples. Considering the wide range of possible binders for the production of stable Eu3+ MOFs, it is possible to design a very specific and selective marking code, which may assist in the identification of ammunition. Although this work has been developed in a forensic context for the codification of ammunition, the methodology here presented can be applied to any situation that requires the differentiation of MOFs containing Eu3+ - which means, used in a variety of situations that require selective marking.sup It is possible to conclude that the insertion of luminescent markers will enable a reliable identification of GSR from Non-Toxic Ammunition residues. Up to date, there are no validated analytical methodologies available that can unequivocally confirm the presence of this type of evidence in a crime scene. Additionally, the proposed ammunition encoding process provides information that is not acquired with the implemented methodologies for conventional ammunition, which makes the proposed method a powerful analytical tool for the investigation of crimes with firearms.
Corresponding Author LIMA, Chemistry Institute, University of Brasília-UNB, P.O. Box 04478, 70904-970 Brasília, Brazil. Phone.: +55 61 3107-3898. E-mail addresses:
[email protected],
[email protected] (I.T. Weber).
ACKNOWLEDGMENTS
Figure 7 (A) VIP scores of PLS-DA built for the fused data. Prediction for training and test sets for each class: (B) Eu(BDC), (C) Eu(BTC) and (D) Eu(DPA).
CONCLUSION Considering a scenario in which there is a selective marking of ammunition for tracking purposes, a fast and non-destructive methodology for GSR classification was proposed based on the insertion of luminescent markers in Non-Toxic ammunition. In order to differentiate and classify three different Eu3+ based markers with different organic ligands, Eu(DPA), Eu(BTC) and Eu(BDC) were synthesized and used as luminescent markers in ammunition. Luminescence, normal and resonance Raman spectroscopies were employed for marker discrimination associated with chemometric tools. While resonance Raman showed potential for the application, the normal Raman profiles suffered high influence from the substrate and other OGSR compounds present. Both luminescence spectral profiles, excitation and emission, also provided complementary information regarding marker discrimination. Therefore, a mid-level data-fusion
The authors would like to acknowledge the funding agencies INCTAA (Processes nº.: CNPq 573894/2008-6; FAPESP 2008/57808-1), NUQAAPE –FACEPE (APQ-0346-1.06/14), Núcleo de Estudos em Química Forense – NEQUIFOR (CAPES AUXPE 3509/2014, Call PROFORENSE 2014), CNPq (428891/2018-7), FACEPE (BFP-0800-1.06/17) and CAPES. We thank the forensic experts Eduardo Makoto Sato and Ronei Maia Salvatori from the National and Institute of Criminalistics – Brazilian Federal Police (NIC/BFP) for research collaboration. The English text of this paper has been revised by Sidney Pratt, Canadian, MAT (The Johns Hopkins University), RSAdip - TESL (Cambridge University).
REFERENCES (1) Brozek-Mucha, Z. Distribution and Properties of Gunshot Residue Originating from a Luger 9 Mm Ammunition in the Vicinity of the Shooting Gun. Forensic Sci. Int. 2009, 183 (1–3), 33–44. (2) Doty, K. C.; Muro, C. K.; Bueno, J.; Halámková, L.; Lednev, I. K. What Can Raman Spectroscopy Do for Criminalistics? J. Raman Spectrosc. 2016, 47 (1), 39–50. (3) Abrego, Z.; Ugarte, A.; Unceta, N.; Fernández-Isla, A.; Goicolea, M. A.; Barrio, R. J. Unambiguous Characterization of Gunshot Residue Particles Using Scanning Laser Ablation and Inductively Coupled Plasma-Mass Spectrometry. Anal. Chem. 2012, 84, 2402– 2409. (4) Lucena, M. A. M.; Oliveira, M. F. L.; Arouca, A. M.; Talhavini, M.; Ferreira, E. A.; Alves, S.; Veiga-Souza, F. H.; Weber, I. T. Application of the Metal-Organic Framework [Eu(BTC)] as a Luminescent Marker for Gunshot Residues: A Synthesis, Characterization, and Toxicity Study. ACS Appl. Mater. Interfaces 2017, 9 (5), 4684–4691. (5) Rendle, D. F. Advances in Chemistry Applied to Forensic Science. Chem. Soc. Rev. 2005, 34 (12), 1021. (6) Blakey, L. S.; Sharples, G. P.; Chana, K.; Birkett, J. W. Fate and Behavior of Gunshot Residue-A Review. J. Forensic Sci. 2017, No. 16, 1–11.
ACS Paragon Plus Environment
Page 9 of 10 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60
Analytical Chemistry
(7) Maitre, M.; Kirkbride, K. P.; Horder, M.; Roux, C.; Beavis, A. Current Perspectives in the Interpretation of Gunshot Residues in Forensic Science: A Review. Forensic Sci. Int. 2017, 270, 1–11. (8) Saverio Romolo, F.; Margot, P. Identification of Gunshot Residue: A Critical Review. Forensic Sci. Int. 2001, 119 (2), 195–211. (9) Melo, L. G. A.; Martiny, A.; Pinto, A. L. Nano Characterization of Gunshot Residues from Brazilian Ammunition. Forensic Sci. Int. 2014, 240, 69–79. (10) NIOSH. Reducing Exposure to Lead and Noise at Indoor Firing Ranges. Niosh. 2010. (11) Martiny, A.; Campos, A. P. C.; Sader, M. S.; Pinto, M. A. L. SEM/EDS Analysis and Characterization of Gunshot Residues from Brazilian Lead-Free Ammunition. Forensic Sci. Int. 2008, 177 (1). (12) Dalby, O.; Butler, D.; Birkett, J. W. Analysis of Gunshot Residue and Associated Materials - A Review. J. Forensic Sci. 2010, 55 (4), 924–943. (13) Goudsmits, E.; Sharples, G. P.; Birkett, J. W. Recent Trends in Organic Gunshot Residue Analysis. Trends Anal. Chem. 2015, 74, 46– 57. (14) Bueno, J.; Lednev, I. K. Raman Microspectroscopic Chemical Mapping and Chemometric Classification for the Identification of Gunshot Residue on Adhesive Tape. Anal. Bioanal. Chem. 2014, 406 (19), 4595–4599. (15) López-López, M.; Delgado, J. J.; García-Ruiz, C. Ammunition Identification by Means of the Organic Analysis of Gunshot Residues Using Raman Spectroscopy. Anal. Chem. 2012, 84 (8), 3581–3585. (16) Ortega-ojeda, F. E.; Torre-Roldána, M.; C. García-Ruiz. Short Wave Infrared Chemical Imaging as Future Tool for Analysing Gunshot Residues Patterns in Targets. Talanta 2017, 167 (December 2016), 227–235. (17) Benito, S.; Abrego, Z.; Sánchez, A.; Unceta, N.; Goicolea, M. A.; Barrio, R. J. Characterization of Organic Gunshot Residues in LeadFree Ammunition Using a New Sample Collection Device for Liquid Chromatography-Quadrupole Time-of-Flight Mass Spectrometry. Forensic Sci. Int. 2015, 246, 79–85. (18) Cardinetti, B.; Ciampini, C.; D’Onofrio, C.; Orlando, G.; Gravina, L.; Ferrari, F.; Di Tullio, D.; Torresi, L. X-Ray Mapping Technique: A Preliminary Study in Discriminating Gunshot Residue Particles from Aggregates of Environmental Occupational Origin. Forensic Sci. Int. 2004, 143 (1), 1–19. (19) Berk, R. E. Automated SEM/EDS Analysis of Airbag Residue. II: Airbag Residue as a Source of Percussion Primer Residue Particles. J. Forensic Sci. 2009, 54 (1), 69–76. (20) Grima, M.; Butler, M.; Hanson, R.; Mohameden, A. Firework Displays as Sources of Particles Similar to Gunshot Residue. Sci. Justice 2012, 52 (1), 49–57. (21) Brozek-Mucha, Z. Chemical and Physical Characterisation of Welding Fume Particles for Distinguishing from Gunshot Residue. Forensic Sci. Int. 2015, 254, 51–58. (22) Polovková, J.; Šimonič, M.; Szegényi, I. Study of Gunshot Residues from Sintox Ammunition Containing Marking Substances. Egypt. J. Forensic Sci. 2015, 5 (4), 174–179. (23) Weber, I. T.; Melo, A. J. G. D.; Lucena, M. a. D. M.; Rodrigues, M. O.; Junior, S. A. High Photoluminescent Metal À Organic Frameworks as Optical. Anal. Chem. 2011, 4720–4723. (24) Melo Lucena, M. A.; Rodrigues, M. O.; Gatto, C. C.; Talhavini, M.; Maldaner, A. O.; Alves, S.; Weber, I. T. Synthesis of [Dy(DPA)(HDPA)] and Its Potential as Gunshot Residue Marker. J. Lumin. 2016, 170, 697–700. (25) Arouca, a. M.; Lucena, M. a. M.; Rossiter, R. J.; Talhavini, M.; Weber, I. T. Use of Luminescent Gunshot Residues Markers in Forensic Context—Part II. Forensic Sci. Int. 2017, 281, 161–170. (26) Allendorf, M. D.; Bauer, C. a; Bhakta, R. K.; Houk, R. J. T. Luminescent Metal-Organic Frameworks. Chem. Soc. Rev. 2009, 38 (5), 1330–1352. (27) Binnemans, K. Interpretation of Europium(III) Spectra. Coord. Chem. Rev. 2015, 295, 1–45. (28) Lucena, M. a. M.; Arouca, a. M.; Talhavini, M.; Alves-Júnior, S.; Weber, I. T. Ammunition Encoding by Means of Co-Doped Luminescent Markers. Microchem. J. 2019, 145, 539–546. (29) Carvalho, M. A.; Talhavini, M.; Pimentel, M. F.; Amigo, J. M.; Pasquini, C.; Junior, S. A.; Weber, I. T. NIR Hyperspectral Images for Identification of Gunshot Residue from Tagged Ammunition. Anal. Methods 2018, 10 (38), 4711–4717. (30) Kumar, R.; Sharma, V. Chemometrics in Forensic Science. Trends Anal. Chem. 2018, 105, 191–201.
(31) Brereton, R. G. Pattern Recognition in Chemometrics. Chemom. Intell. Lab. Syst. 2015, 149, 90–96. (32) Wold, S.; Sjöström, M.; Eriksson, L. PLS-Regression: A Basic Tool of Chemometrics. Chemom. Intell. Lab. Syst. 2001, 58 (2), 109– 130. (33) Ballabio, D.; Consonni, V. Classification Tools in Chemistry. Part 1: Linear Models. PLS-DA. Anal. Methods 2013, 5 (16), 3790. (34) Bro, R.; Smilde, A. Principal Component Analysis. Anal. Methods 2014, No. 6, 2812–2831. (35) Brereton, R. G. Pattern Recognition 5.1. 2007. (36) Wold, S.; Sjostrom, M.; Eriksson, L.; Sweden˚˚, S. . Chemom. Intell. Lab. Syst. 2001, 58, 2001–2109. (37) Borràs, E.; Ferré, J.; Boqué, R.; Mestres, M.; Aceña, L.; Busto, O. Data Fusion Methodologies for Food and Beverage Authentication and Quality Assessment - A Review. Anal. Chim. Acta 2015, 891, 1–14. (38) Silvestri, M.; Elia, A.; Bertelli, D.; Salvatore, E.; Durante, C.; Li Vigni, M.; Marchetti, A.; Cocchi, M. A Mid Level Data Fusion Strategy for the Varietal Classification of Lambrusco PDO Wines. Chemom. Intell. Lab. Syst. 2014, 137, 181–189. (39) Wold, S.; Esbensen, K.; Geladi, P. Principal Component Analysis. Chemom. Intell. Lab. Syst. 1987, 2, 37–52. (40) Brereton, R. G.; Lloyd, G. R. Partial Least Squares Discriminant Analysis: Taking the Magic Away. J. Chemom. 2014, 28 (4), 213–225. (41) Geladi, P.; Kowalski, B. R. Partial Least-Squares Regression: A Tutorial. Anal. Chim. Acta 1986, 185, 1–117. (42) Yamamoto, S.; Bouř, P. Detection of Molecular Chirality by Induced Resonance Raman Optical Activity in Europium Complexes. Angew. Chemie - Int. Ed. 2012, 51 (44), 11058–11061. (43) Koningstein, J. A. Advances in Electronic Raman Spectroscopy of the Lanthanides. Russ. Chem. Rev. 1973, 42 (10), 834– 850. (44) Farrés, M.; Platikanov, S.; Tsakovski, S.; Tauler, R. Comparison of the Variable Importance in Projection (VIP) and of the Selectivity Ratio (SR) Methods for Variable Selection and Interpretation. J. Chemom. 2015, 29 (10), 528–536. (45) Serwy, I. B. et al. [Ln 2 (BDC) 3 (H 2 O) 4 ] n : A low cost alternative for GSR luminescent marking. J. Lumin. 2018, 200, 24–29.
ACS Paragon Plus Environment
Analytical Chemistry 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60
ACS Paragon Plus Environment
Page 10 of 10