Smartphone Cortex Controlled Real-Time Image Processing and

Sep 26, 2017 - We present the application of a smartphone anatomy based technology in the field of liquid phase bioseparations, particularly in capill...
0 downloads 6 Views 3MB Size
Technical Note pubs.acs.org/ac

Smartphone Cortex Controlled Real-Time Image Processing and Reprocessing for Concentration Independent LED Induced Fluorescence Detection in Capillary Electrophoresis Mate Szarka† and Andras Guttman*,†,‡ †

Horváth Csaba Laboratory of Bioseparation Sciences, Research Centre for Molecular Medicine, Faculty of Medicine, University of Debrecen, H-4032 Debrecen, 98. Nagyerdei krt., Hungary ‡ MTA-PE Translational Glycomics Laboratory, University of Pannonia, Veszprem, Hungary ABSTRACT: We present the application of a smartphone anatomy based technology in the field of liquid phase bioseparations, particularly in capillary electrophoresis. A simple capillary electrophoresis system was built with LED induced fluorescence detection and a credit card sized minicomputer to prove the concept of real time fluorescent imaging (zone adjustable time-lapse fluorescence image processor) and separation controller. The system was evaluated by analyzing under- and overloaded aminopyrenetrisulfonate (APTS)labeled oligosaccharide samples. The open source software based image processing tool allowed undistorted signal modulation (reprocessing) if the signal was inappropriate for the actual detection system settings (too low or too high). The novel smart detection tool for fluorescently labeled biomolecules greatly expands dynamic range and enables retrospective correction for injections with unsuitable signal levels without the necessity to repeat the analysis.

S

photon-counting module from a controller (PC). Avalanche photodiode counting modules only achieve high sensitivity when the optics are assembled so the emitted light is centered at the tiny active area on the diode, thus, requiring a precise optical geometry.7 PMTs are typical choices for most common applications dealing with pico-nanoliter size focal volumes,8 but they have the same drawbacks as the aforementioned detector types (i.e., the initial detector settings determine dynamic range of the detector during the analysis). Recent advances in smartphone technology (imaging, processing, etc.) offer currently unexploited new options to improve the performance of light detection based analytical and bioanalytical devices9,10 At present, most commercial capillary electrophoresis and liquid chromatography systems employ either photodiodes or PMTs as core part of their optical detection setting. In the case of laser-induced fluorescent detection these require large accessories, including laser modules, precise electronics, stable power supplies, external detectors, and signal converters. New advances in LED technology,11 microcontroller platforms,12 and CMOS or CCD based optical detectors13 offer similar detection efficiency as those traditional techniques, but with significantly smaller designs, cheaper price tags, and most importantly, post-task data processing capabilities when associated with smartphone cortex-based architecture.14 In these devices, the intensity profiles are not stored as single values at a given time point, but

emiconductor based devices are the core of contemporary, state of the art optical detectors for liquid phase separation systems.1 When these modules capture emitted or transmitted photons, they produce the intensity response in the separation runs (i.e., the electropherograms in the case of capillary electrophoresis). Diode array detectors (DAD) are commonly used in spectroscopy based detection units in liquid chromatography (LC) and capillary electrophoresis (CE).2 DAD detects UV to VIS region light absorption with an array of photodiodes to collect information over a wide range of wavelengths within the same time window as conventional single wavelength UV−vis detectors. The full spectrum is usually taken at intervals of given frequency during the separation. If the electropherogram is collected at a specific wavelength the sample components are identified by their migration time only. Tiny migration time deviations can greatly complicate component identification. A great advantage of DAD detection is that components can be identified by their corresponding UV−vis spectra from relevant databases.3 DAD detection works similarly to CCD or CMOS4 detectors in terms of multispectrality, but without the capability of storing the generated images as raw data for post-task reprocessing. Coupling fluorescence detection to CE achieves high sensitivity and selective detection of fluorescently labeled molecules.5 Fluorescent molecules vary in their unique absorption (excitation) and emission profile, and wavelengths must be carefully selected for maximal detection sensitivity. With femtoliter scale focal volumes (probe size) photon-counting devices can be utilized because of their low background noise.6 However, these systems require additional interfaces to run the © XXXX American Chemical Society

Received: August 29, 2017 Accepted: September 14, 2017

A

DOI: 10.1021/acs.analchem.7b03525 Anal. Chem. XXXX, XXX, XXX−XXX

Technical Note

Analytical Chemistry

Sample Preparation. IgG N-glycans and the maltooligosaccharide (MD) ladder were prepared and APTS labeled following the process earlier published by Varadi et al.18 The IgG samples (Sigma, St. Louis, U.S.A.) were prepared using different starting material concentrations of 2 and 0.2 mg/mL. This latter resulted in generally poor signal outcome, simulating in this way an error in the sample preparation process. The MD ladder sample (SCIEX, Brea, CA), on the other hand, was injected under overloading conditions (35 mg/mL APTS labeled MD ladder) simulating possible evaporation mediated increase in sample concentration, for example, when an open 96-well plate is used during large-scale analysis. For limit of detection (LOD) determination, 40 mM APTS was dissolved in 15% acetic acid and diluted with DDI water as specified under the relevant sections of this paper (S/N ratio section). The N−CHO separation gel buffer (SCIEX) was used for all CE-LedIF separations.

rather as an array of values (a digital image, with the detector zone of the capillary in the field of view) at any given moment during the separation. Image processing algorithms executed over the time-lapse image sequences can virtually decrease or increase the intensity profile in the electropherogram without the need to reinject and reanalyze the sample of interest, thus readily support rapid and/or high throughput applications. Consequently, the novel smartphone cortex controlled realtime image processing and reprocessing based approach introduced in this paper offers the opportunity to utilize state of the art hardware and software solutions for expending the selectivity and the dynamic range for capillary electrophoresis and other liquid phase separation methods with LED induced fluorescence detection.15



EXPERIMENTAL SECTION

Optics. The 50 μm ID bare fused silica separation capillary (Polymicro Technologies, Phoenix, AZ) was used with 20 cm effective length (30 cm total). For image acquisition based detection, the probe zone was placed in the focal plane of a custom-made 3D printed epifluorescence lens setting, based on the recent report of Prikryl et al.16 Illumination was provided by an OSB5XME3E1E blue LED. An Optolong BP460−490 excitation filter was used at the illumination side (Yulong Optics Co, Ltd., Kunming, PRC) along with a DM505 dichroic mirror (Optolong) to ensure illumination and detection through a single 20x objective (Carl Zeiss, Jena, Germany). Thus, the emission channel only allowed transmission of green light through the BA510−550 band-pass filter and the dichroic mirror, considering the Stokes shift of the aminopyrenetrisulfonate (APTS, Sciex, Brea, CA) dye (λex = 455 nm, λem = 510 nm) used for labeling the carbohydrate samples. High Voltage Power Supply (HVPS). A Spellman (New York, U.S.A.) M series −20 kV DC power supply was used, controlled by a custom-made circuit shield (Vitrolink, Debrecen, Hungary). Host. In order to provide precise control over the sample injection process, separation and imaging (at the capillary probe zone) an ARM based Smartphone cortex architecture Raspberry Pi-3 minicomputer was used as central controller unit (host). Injection and separation steps were both accomplished by custom python scripts, executed by the minicomputer using Raspbian (Raspberry Pi) operating system. The host was reached through an SSH protocol from the client machine and also served as preprocessor component for the fluorescence detector during the analysis. Detector. A 5 megapixel RpiCam NoIR type 1, CCD camera (Raspberry Pi) was utilized with its original lenses removed. The CCD was mounted on the ocular lens, which was placed in the back focal plane of the objective.16 Detection was accomplished by using the Raspistill library (Raspberry Pi) in time-lapse mode from the SSH terminal, Putty (Simon Tatham, Cambridge, U.K.). Images were collected in bmp or jpeg formats. Client. Remote injection, separation, image acquisition control, and data processing were all executed by the client machine, running under Windows 10 (Microsoft, Seattle, U.S.A.). The scripts were written in Matlab (MathWorks Inc., Natick, U.S.A.) and ImageJ Fiji17 macro languages. Additional MIJ library (Biomedical Imaging Group, Lausanne, Switzerland) was used to support the interoperation between the Fiji and Matlab softwares.



RESULTS AND DISCUSSION Currently used fluorescence detectors for liquid separation techniques are nonresponsive to signals that fall outside of the quantifiable detection range (either too high or too low). Therefore, samples that fall below detection limits or cause detector saturation must be reinjected with properly adjusted detector settings and/or sample concentrations, until the correct range is established. The smartphone cortex driven imaging based approach introduced in this paper adequately addresses these issues by performing real-time image processing of the fluorescence signal. Image Pre- and Post-Task Processing. During image preprocessing, the client periodically checked for and downloaded any new, unprocessed images from the host. All new images were analyzed, filtered and saved. During image posttask processing, the custom Fiji/ImageJ macros generated the mean pixel intensity value outputs of the input bitmap (or jpeg) images “on the fly”. Subtraction of the background (single noise frame) image was carried out before the calculation of the mean pixel values in every incoming image from the host, thereby reducing the noise from the image sequences. The filtered mean image sequence pixel values were visualized as graphs (electropherograms) in real-time. The values were automatically saved in ASCII text format as the processing continued. S/N Ratio. During the separation of the APTS labeled maltooligosaccharide ladder, the signal-to-noise ratio was enhanced as follows. The RGB images from frame 40 until the APTS peak were treated as noise. This ∼200 frame sequence (dead-time) was ideal for noise level determination. A single background image was calculated by averaging all noise frames from four individual ladder separations. To determine the limit of detection (LOD), 3 runs of each of the following dilutions of the 40 mM APTS stock solution were averaged: 3 of 400 pM, 40 pM, and 4 pM monitoring the data from the green channel. The data generated by the red and blue channels were discarded. The highest intensity peak expected at frame number ∼260 (∼3 min) in 11 × 12 size pixel2 (detection probe zone) images was the APTS peak determined from the higher concentration runs of 4 nM, 40 nM, etc. The averaging process clearly produced a pure signal by eliminating noise or pixel artifacts and setting a steady baseline. The relative fluorescent units (RFU) were 0.4924, 0.2197, and 0.0789 for the 400 pM, 40 pM and 4 pM APTS dilutions, respectively. The logarithmic dilution versus RFU is delineated by eq 1, with (R2 = 0.999): B

DOI: 10.1021/acs.analchem.7b03525 Anal. Chem. XXXX, XXX, XXX−XXX

Technical Note

Analytical Chemistry

Figure 1. Virtual adjustment of the capillary probe. (A) Images of the capillary probe (sections a−f) and virtually adjusted probe (sections a′−f′); (B) Histograms of whole images (sections a−f) and ROIs (sections a′−f′) of the capillary; (C) Signal intensity plot of the whole image (sections a− f) and virtually adjusted capillary probe (sections a′−f′).

y = −0.378 ln(x) + 0.4896

(1)

where y = logarithmic dilution value and x = measured signal intensity. C

DOI: 10.1021/acs.analchem.7b03525 Anal. Chem. XXXX, XXX, XXX−XXX

Technical Note

Analytical Chemistry

Figure 2. Histograms of the original (A) and processed (B) IgG glycoprotein FA2 structure as well as the overloaded (C) and restored (D) maltopentose peaks extracted from the electropherograms generated by the CE-LedIF instrument. Horizontal axis represents the measured pixel intensities (8-bit scale): 0, dark image; 255, saturated or white image. Vertical axis stands for the intensity values taken by the pixels of the image (count).

Figure 3. Capillary electrophoresis - LED induced fluorescent detection (CE-LedIF) analysis of APTS labeled IgG N-glycans (A) and a set of maltooligosaccharides (B). (A) Lower trace: Injection of a heavily diluted IgG glycan sample (starting material 0.2 mg/mL IgG). Upper trace: Enhanced signal intensity after Positive Histogram Value Displacement (PHVD) processing. The empty star depicts the core fucosylated agalactosylated biantennary structure (FA2). (B) Lower trace: overloaded sample injection (35 mg/mL initial sugar concentration); Upper trace: Processed signal after Negative Histogram Value Displacement (NHVD). The star depicts the maltopentaose component of the maltooligosaccharide ladder sample. Conditions: N−CHO background electrolyte, 20 cm (effective length, 30 cm total) 50 μm ID bare fused silica capillary; Injection: 5 kV for 3 s; Applied voltage: 12 kV (cathode at the injection side); Separation temperature: ambient.

Virtually Adjusting the Capillary Detection Probe Size in the Images. In digital imaging, the signal processed from the CCD or CMOS camera mostly provides pixel arrays that contain the intensity information on the recorded object with a specific pixel resolution. This information is mostly encoded at 8−32 bits in case of RGB images. On a 24 bit encoded image, each channel (red, green, and blue) can contain a value from 0 to 255 per pixel. Zero values in all pixels represent a black image, while 255 in all pixels a saturated or white image. Counting the values of all the pixels for a full or partial image resulted in a histogram, which contained essential information about the given 2-dimensional pixel array or image (i.e., minimum pixel values, maximum pixel values, mean - average

pixel values, their distribution, etc.). Recalculating the histogram of the region of interest (ROI) instead of the whole image can create significant shifts in pixel value range distribution, since the number of processed pixels in the image change accordingly. Shown in Figure 1A, panels a−f, is an image sequence as created by our CE-LedIF setup. The vertical bright line in the middle is the excited capillary probe with APTS sample in the capillary. Their passing can be measured by processing the histogram of each image (time-lapse sequence) during the separation as delineated in panels a−f in Figure 1B. However, if the ROI (e.g., the probe size of the capillary) changes, as shown in Figure 1B, panels a′−f′, the histogram changes accordingly. Plotting the average of the image D

DOI: 10.1021/acs.analchem.7b03525 Anal. Chem. XXXX, XXX, XXX−XXX

Technical Note

Analytical Chemistry histogram values (mean = y axis) versus time (image sequence = x axis) provides the peaks in the electropherogram (Figure 1C). Correctly choosing the ROI size and shape over the entire image sequence can increase or decrease the signal intensity without distortion. A similar effect can be achieved by pixel value adjustment instead of ROI adjustment. In case of inappropriate (too low or too high) sample concentration, the negative (NHVD: Negative Histogram Value Displacement) or positive (PHVD: Positive Histogram Value Displacement) window/ level adjustment relative to the original histogram of the sample was executed depending on the nature of the problem. The algorithms altered the “window” (range of minimum and maximum pixel values) and “level” (position of that range in greyscale or 8-bit intensity space) of the image sequences (electropherogram). In image processing, LUTs or lookup tables provided a single output value for each of the index value ranges. In other words, the lookup table contained equivalents to convert brightness within an image into numbers. In an 8-bit gray scale image, there were 256 values. The black was set to 0, the white was 255, and all of the other intensity gradients were given values between them. During window/level operation, a linear gray scale transform function was applied on the lookup table (LUT) of the image. This LUT was defined by two parameters, “window” and “level”. “Window” was a nonzero slope over the LUT. “Level” was the center of the segment bracketed by the “window”. As a result, the desired pixel intensities (concentration) belonging to a subset of the whole intensity range were extended lineary to those pixels that were outside the specified range. Every pixel in the input image i(x,y) got copied to the corresponding pixel in the output image o(x,y), but the new pixel intensity value in the output image was defined by the newly - window/level - adjusted LUT:

o(x,y) = LUT[i(x,y)]

using Fiji/ImageJ. First, the red and blue channels were added by the following arithmetic operation: O[i] = R[i] + B[i]

(3)

where O represented the result image of the “Add” operation, R = red channel image, B = blue channel image, and [i] = corresponding image. The output image sequence was further processed by another operon called “Transparent-zero” located in the ImageJ/Fiji Image Calculator submenu. Pixels of the green channel image sequence were overlapped with the result sequence of the previous O “Add” operon and the pixels of the latter one were set to transparent. All operations were performed with 32-bit floating point conversion. Thereby, all the pixels were counted in the extended intensity scale by the so-called NHVD algorithm that combined the RGB channels of the image sequences (Figure 2B, mean intensity = 241), resulting in adequate intensity profile as shown in Figure 3B, upper trace. Saturating peaks in the electropherogram of the MD ladder complicate defining glucose unit values that are essential for glycan structure elucidation.19 Propagating the modified histogram over the entire raw image sequence produced annotatable electropherograms in the required intensity−and therefore−concentration ranges, as demonstrated by the upper traces in Figure 3A,B. The capillary electrophoresis - LED induced fluorescent detection (CE-LedIF) traces of heavily diluted APTS-labeled IgG N-glycans were investigated. As Figure 3A depicts, these apparently did not reveal any evaluable peaks due to the low injected sample concentration, which was under the set detection limit. Executing the Positive Histogram Value Displacement (PHVD) post-task algorithm centered to 10 intensity values with 2 intensity values wide “Window” parameters over the raw image sequence, the reprocessed electropherogram (upper trace in Figure 3A) clearly showed all major peaks, despite of the very low sample concentration injected. Conversely, image processing with overloaded sample injection was evaluated by analyzing the APTS labeled maltooligosaccharide ladder. The lower trace in Figure 3B exhibits such an overloaded profile. The Negative Histogram Value Displacement (NHVD) post-task algorithm was executed over the raw image sequence reestablishing a readily evaluable and quantifiable profile of the MD ladder as shown by the upper trace in Figure 3B. RFU Adjustment. The change in signal intensity can be explained by the working principles of the hardware and the algorithms. In case of conventional pretuned single point analogue photodiode detectors, the acquired intensity range is practically static. Using the methods introduced in this work, fluorescence detection became array-like, providing a maximum of 4915200 data points (an array of mini detectors) in a rectangle shape (digital image) due to the properties of the CCD camera. Thus, the light intensity was dynamically collected, consequently, enabling retrospective fine-tuning of the detector signal.

(2)

Thereby, the input pixels with intensities that fell into the range of interest (concentration range or brightness of the fluorescent signal) were selectively contrast extended in the output image and the rest were discarded. Figure 2A,B depicts the window/level adjusted histogram of the peak corresponding to the FA2 structure (marked with an empty star in Figure 3A). The original, heavily diluted sample peak moved from the low pixel intensity counts as undetectable structure in Figure 3A, lower trace (mean intensity = 3.4; image not shown), toward higher intensity values with stronger distribution due to window/level parameter adjustment (PHVD algorithm) resulting in enhanced signal of the structure in Figure 3A, upper trace (mean intensity = 53.4). Such an action can only be processed if the detection probe zone in the capillary, that is, the intensity range of an array is virtually adjustable (in the image sequence with the detector zone in the field of view). Another example is shown in Figure 3B where the MD ladder at lower trace represented the overloaded condition by examining a saturated 56 × 75 pixel2 (brighter image, not shown) area of the CCD chip. Note that the original maltopentaose structure (DP5) signal (marked with the black star in Figure 3B, lower trace) is indistinguishable from the other peaks since the intensities (all peaks until DP8 in Figure 3B, lower trace) exceeded the preset highest scale (Figure 2A; mean intensity = 252.1). Therefore, to resolve this issue, in addition to the green channel, the other two remaining channels (red and blue) were processed by image calculation



CONCLUSIONS Implementation of smartphone cortex technology allowed zone adjustable time-lapse imaging based detection of the electrophoretic separation of fluorescently labeled (APTS) oligosaccharides via this novel image collection/image processing technique. Two algorithms were executed during post-task processing depending on the concentration of the injected E

DOI: 10.1021/acs.analchem.7b03525 Anal. Chem. XXXX, XXX, XXX−XXX

Analytical Chemistry



samples. Instead of the analogue intensity values that are traditionally produced by conventional fluorescent detectors, the approach presented in this paper provided the ability to acquire a vast amount of information from the detection signal that would be normally impossible with the use of photodiodes or other conventional detection systems. Applying the virtual detection probe (zone) adjustment option, analysis of precious samples, like limited biological specimens (biopsy, circulating tumor cells, formalin-fixed, paraffin-embedded tissue, etc.) or samples either investigated in Space or brought back to Earth can compensate for human errors. It can also compensate naturally occurring phenomena, such as evaporation (resulting in overconcentrated samples) during continuous measurement of hundreds of samples without the necessity of repeating any of the measurements. Similar technology can be used to determine individual, and unknown structures19 during the analysis by continually updating structural annotations from cloud databases even before the separation is finished. Again, implementing such image processing based detectors will alleviate the need of time-consuming and labor intensive repetitive analysis due to inappropriate detection settings or incorrectly chosen (too low or too high) sample concentrations in liquid phase separation techniques.



Technical Note

REFERENCES

(1) Johnson, M. E.; Landers, J. P. Electrophoresis 2004, 25, 3513− 3527. (2) Heiger, D. N.; Kaltenbach, P.; Sievert, H. J. Electrophoresis 1994, 15, 1234−1247. (3) Lambert, W. E.; Van Bocxlaer, J. F.; De Leenheer, A. P. J. Chromatogr., Biomed. Appl. 1997, 689, 45−53. (4) Timperman, A. T.; Khatib, K.; Sweedler, J. V. Anal. Chem. 1995, 67, 139−144. (5) Rossi, T. M. J. Pharm. Biomed. Anal. 1990, 8, 469−476. (6) Eisaman, M. D.; Fan, J.; Migdall, A.; Polyakov, S. V. Rev. Sci. Instrum. 2011, 82, 071101. (7) Dada, O. O.; Essaka, D. C.; Hindsgaul, O.; Palcic, M. M.; Prendergast, J.; Schnaar, R. L.; Dovichi, N. J. Anal. Chem. 2011, 83, 2748−2753. (8) Felhofer, J. L.; Blanes, L.; Garcia, C. D. Electrophoresis 2010, 31, 2469−2486. (9) Zhu, H.; Ozcan, A. Methods Mol. Biol. 2015, 1256, 171−190. (10) Mauk, M. G.; Liu, C.; Song, J.; Bau, H. H. Microarrays 2015, 4, 474−489. (11) Nakamura, S. Angew. Chem., Int. Ed. 2015, 54, 7770−7788. (12) Sachdeva, P. K. S. Int. J. Curr. Eng. Technol. 2014, 4, 2. (13) Joardar, S. C. A.; Rakshit, A. IEEE Trans. Instrum. Meas. 2015, 64, 7. (14) Schneider, C. A.; Rasband, W. S.; Eliceiri, K. W. Nat. Methods 2012, 9, 671−675. (15) Wu, X. Z.; Pawliszyn, J. Electrophoresis 2004, 25, 3820−3824. (16) Prikryl, J.; Foret, F. Anal. Chem. 2014, 86, 11951−11956. (17) Schindelin, J.; Arganda-Carreras, I.; Frise, E.; Kaynig, V.; Longair, M.; Pietzsch, T.; Preibisch, S.; Rueden, C.; Saalfeld, S.; Schmid, B.; Tinevez, J. Y.; White, D. J.; Hartenstein, V.; Eliceiri, K.; Tomancak, P.; Cardona, A. Nat. Methods 2012, 9, 676−682. (18) Varadi, C.; Lew, C.; Guttman, A. Anal. Chem. 2014, 86, 5682− 5687. (19) Jarvas, G.; Szigeti, M.; Chapman, J.; Guttman, A. Anal. Chem. 2016, 88, 11364−11367.

AUTHOR INFORMATION

Corresponding Author

*E-mail: [email protected]. Fax:+36 52 255 539. ORCID

Mate Szarka: 0000-0003-2670-5554 Andras Guttman: 0000-0001-5136-8479 Notes

The authors declare no competing financial interest.



ACKNOWLEDGMENTS The authors gratefully acknowledge the support of the Momentum (Lendulet) Grant #97101 of the Hungarian Academy of Sciences (MTA-PE Translational Glycomics Group), the NKFIH (K 116263) Grant of the Hungarian Government, the BIONANO_GINOP-2.3.2-15-2016-00017 Project, and the kind assistance of Marton Szigeti and Balazs Reider (this is contribution #133 from the Horváth Csaba Laboratory of Bioseparation Sciences).



ABBREVIATIONS: APTS = aminopyrenetrisulfonate; BMP = bitmap image; CCD = charge-coupled Device; CE-LedIF = capillary electrophoresis - light emitting dioide induced fluorescence; CMOS = complementary metal−oxide−semiconductor; CTC = circulating tumor cells; FA2 = core fucosylated agalactosylated biantennary IgG sugar structure; FFPE = formalin-fixed, paraffin-embedded tissue; HVPS = high voltage power supply; IgG = immunoglobulin G; JPEG = joint photographic experts group; LED = light emitting diode; LOD = limit of detection; MD = maltooligosaccharide; MP = megapixel; NHVD = negative histogram value displacement; PHVD = positive histogram value displacement; PMT = photomultiplier tube; RFU = relative fluorescent unit; RGB = red green blue color model; ROI = region of interest; SSH = secure shell; S/N = signal to noise ratio F

DOI: 10.1021/acs.analchem.7b03525 Anal. Chem. XXXX, XXX, XXX−XXX