Carbon Nanotube Synaptic Transistor Network for Pattern Recognition

Subscriber access provided by CMU Libraries - http://library.cmich.edu

Article

Carbon Nanotube Synaptic Transistor Network for Pattern Recognition Sungho Kim, Jinsu Yoon, Hee-Dong Kim, and Sung-Jin Choi ACS Appl. Mater. Interfaces, Just Accepted Manuscript • DOI: 10.1021/acsami.5b08541 • Publication Date (Web): 29 Oct 2015 Downloaded from http://pubs.acs.org on October 30, 2015

Just Accepted “Just Accepted” manuscripts have been peer-reviewed and accepted for publication. They are posted online prior to technical editing, formatting for publication and author proofing. The American Chemical Society provides “Just Accepted” as a free service to the research community to expedite the dissemination of scientific material as soon as possible after acceptance. “Just Accepted” manuscripts appear in full in PDF format accompanied by an HTML abstract. “Just Accepted” manuscripts have been fully peer reviewed, but should not be considered the official version of record. They are accessible to all readers and citable by the Digital Object Identifier (DOI®). “Just Accepted” is an optional service offered to authors. Therefore, the “Just Accepted” Web site may not include all articles that will be published in the journal. After a manuscript is technically edited and formatted, it will be removed from the “Just Accepted” Web site and published as an ASAP article. Note that technical editing may introduce minor changes to the manuscript text and/or graphics which could affect content, and all legal disclaimers and ethical guidelines that apply to the journal pertain. ACS cannot be held responsible for errors or consequences arising from the use of information contained in these “Just Accepted” manuscripts.

ACS Applied Materials & Interfaces is published by the American Chemical Society. 1155 Sixteenth Street N.W., Washington, DC 20036 Published by American Chemical Society. Copyright © American Chemical Society. However, no copyright claim is made to original U.S. Government works, or works produced by employees of any Commonwealth realm Crown government in the course of their duties.

Page 1 of 28

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

ACS Applied Materials & Interfaces

Carbon Nanotube Synaptic Transistor Network for Pattern Recognition Sungho Kim1, Jinsu Yoon2, Hee-Dong Kim1 & Sung-Jin Choi2,*

1

Department of Electrical Engineering, Sejong University, Seoul 05006, Korea

2

School of Electrical Engineering, Kookmin University, Seoul 02707, Korea

*Correspondence to S. J. C. ([email protected]).

ACS Paragon Plus Environment

1


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 2 of 28

Abstract Inspired by the human brain, a neuromorphic system combining complementary metaloxide-semiconductor (CMOS) and adjustable synaptic devices may offer new computing paradigms by enabling massive neural-network parallelism. In particular, synaptic devices, which are capable of emulating the functions of biological synapses, are used as the essential building blocks for an information storage and processing system. However, previous synaptic devices based on two-terminal resistive devices remain challenging due to their variability and specific physical mechanisms of resistance change, which lead to a bottleneck in the implementation of a high-density synaptic device network. Here we report that a three-terminal synaptic transistor based on carbon nanotubes can provide reliable synaptic functions that encode relative timing and regulate weight change. In addition, using system-level simulations, the developed synaptic transistor network associated with CMOS circuits can perform unsupervised learning for pattern recognition using a simplified spike-timing-dependent plasticity scheme.

Keywords: analog switching, carbon nanotube, neuromorphic system, pattern recognition, synaptic device, transistor.


2

Page 3 of 28

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


INTRODUCTION The human brain is an exceptionally energy-efficient computing system specialized in perception, recognition, learning, and decision-making. Although modern computers can similarly execute such tasks, they require orders of magnitude more energy and complex programming. Artificial neural networks are inspired by the human brain, and unlike conventional von Neumann-type computers, these networks enable adaptable and high-efficiency computing due to the massive parallelism of neurons and synapses.1,2 For information processing in neural networks, spikes from the pre-synaptic neurons can be transmitted through the synapses and generate a membrane potential and thus pre-synaptic spikes based on the relative strengths of the synapses (i.e., the synaptic weights). These synaptic weights can be modulated by the spikes from pre- and post-synaptic neurons and can be maintained over the long-term; thus, the plasticity of the synaptic weights enables learning and memory operations in the system.3 To date, the functions of neurons and synapses have been emulated by circuits combining complementary metal-oxide-semiconductor (CMOS) and adjustable two-terminal resistive devices (memristors). In particular, critical synaptic-learning rules (i.e., spike-timing-dependent plasticity (STDP)) have been demonstrated by memristors in a number of studies.4-9 Theoretical studies have suggested that STDP can be used for learning in spiking neural networks,10,11 and therefore, memristors have been considered as promising candidates for electrical synapse devices. As discussed in previous studies, however, the sustainability of memristors is still in doubt, particularly with regards to the device-to-device variability that is common to all memristor technologies.12 The specific physical mechanism of resistance change in most prospective metal-oxide-based memristors, which is a reversible atomic-scale modulation of oxygen vacancies,13 is responsible for the unwanted device-to-device reproducibility issue of


3


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 4 of 28

relevant device parameters, which is the primary reason why the only demonstrations of memristor neuromorphic networks have been based on disconnecting each memristor14,15 or the use of low-density crossbar arrays.16 These approaches are incompatible with the goal of achieving extremely high-density neuromorphic networks. In this study, we demonstrate an electronic synapse device based on carbon nanotube (CNT) three-terminal transistors and show that by controlling the internal dynamics of CNT synaptic transistors, reliable long-term synaptic functionality (i.e., STDP) can be achieved. To reduce the synaptic-device variability, we used a pre-separated, semiconducting enriched singled-walled CNT matrix, which potentially enables a large-scale neuromorphic circuit with a high uniformity. We used a system-level simulation based on a simplified STDP scheme for a CNT synaptic transistor network, which shows the potential for unsupervised learning and consequent pattern recognition ability. These simulation results can provide guidelines for continued design and optimization of CNT synaptic transistors to produce a large-scale, neuromorphic computing system.

EXPERIMENTAL SECTION A highly p-doped silicon substrate was used as the back gate with thermally grown, 55-nmthick, back-gate oxide (SiO2). The substrate was first functionalized with a poly-L-lysine solution to form an amine-terminated layer, which acted as an effective adhesion layer for singlewalled CNTs. The substrate was then rinsed with deionized (DI) water and isopropanol. To deposit a random matrix of CNTs, the substrate was immersed in a high-purity, semiconducting, nanotube solution for 1 min, followed by a thorough rinse with DI water and isopropanol and DI water; the substrate was then dried with flowing nitrogen. The 90% semiconducting nanotube


4

Page 5 of 28

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


solutions (IsoNanotubes-S) used in this study were provided by Nanointegris, Inc. The source/drain electrodes were inkjet-printed with silver (Ag, InkTec Tec-IJ-060) nanoparticle ink using an inkjet printer (Unijet UJ200MF) integrated with a piezoelectric type dispenser and 50µm-orifice nozzles. Prior to inkjet printing, Ag ink was filtered through a 5 µm polytetrafluoroethylene (PTFE) syringe filter to avoid aggregation of Ag nanoparticles. The typical inkjet droplet volume ranged from 40 to 50 pL. Subsequently, a 150 °C annealing process was employed. Finally, poly-4-vinylphenol (PVP) was also printed onto the surface to designate the active channel area. A subsequent oxygen plasma-etching step was performed to isolate the devices and remove undesirable leakage paths. The PVP was then sequentially removed with acetone, isopropanol, and flowing nitrogen. The channel lengths and widths of the fabricated device were 170 ~ 480 µm and 300 ~ 490 µm, respectively (microscopy image of the fabricated devices is shown in Supporting Information Note 1). Electrical measurements were conducted using a low-noise Keithley 4200 semiconductor characterization system or a custom-built electrical measurement system in combination with a probe station. During all measurements, the bias voltage was always applied to the gate, the source, and the drain electrodes.

RESULTS AND DISCUSSION The base module of the neuromorphic system reproduces the interrelation of a pre-synaptic neuron connected with other post-synaptic neurons via synapses (Figure 1a). In general, neurons can be easily emulated using an integrate-and-fire (I&F) circuit based on common Si transistors.17,18 However, the emulation of a synapse is relatively challenging due to its unique characteristics, which include being non-volatile and having the ability to modulate its analogue


5


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 6 of 28

state. In this study, the synaptic transistors based on a random matrix of single-walled CNTs are shown to reproduce synaptic functions (Figures 1b and 1c). Pre-separated, semiconducting CNTs (90%) enable high uniformity and sustainability of the synaptic transistors and reliable synaptic functions (see Supporting Information Note 2). In principle, neurons transmit signals thorough the CNT synaptic transistor channel, where the transistor’s gate and source electrodes are connected to the pre-neuron (input neuron) and the post-neuron (output neuron), respectively. The channel conductance of the CNT synaptic transistors can be controlled via the pre-synaptic spike (Vpre), which is applied to the gate, and post-synaptic spike (Vpost-), which is applied to the source. Thus, the variable channel conductance of a CNT synaptic transistor will be used as a synaptic weight. In addition, each CNT synaptic transistor has a shared output channel to a given output neuron, as shown in Figure 1b. Pre-synaptic spikes from input neurons can trigger multiple CNT synaptic transistors simultaneously, and all post-synaptic currents generated from CNT synaptic transistors are collected and accumulated at an output neuron (by I&F circuits). In detail, input pre-synaptic spikes represent sensing signals from the external environment. The timing and rate of these spikes represent the analogue information of the sensing signals from the sensors. These input spikes lead to post-synaptic currents, which are determined based on the channel conductance of each CNT synaptic transistor. Then, all post-synaptic currents are accumulated at an output neuron, which fires post-synaptic spikes back to the CNT synaptic transistor if the accumulated post-synaptic current level is above a given threshold value. Based on the relative timing of the pre- and post-synaptic spikes, the synaptic weight can be modulated to any analogue state and memorized over the long-term.


6

Page 7 of 28

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


First, as mentioned above, the synaptic weight can be reproduced by an intrinsic analogue state of channel conductance in the CNT synaptic transistor. When the gate voltage (VG) was swept from -7 V to +5 V and back to -7 V, a hysteresis of the source current (IS) under a constant drain voltage (VD) was observed (Figure 2a); a positive VG increased the channel conductance, which is defined as the ‘potentiation’ of the synaptic weight, and a negative VG decreased the conductance, which is defined as the ‘depression’ of the synaptic weight. This hysteresis indicates an intrinsic characteristic of the variable channel conductance in the CNT synaptic transistor that is dependent on the control of VG. The hysteresis in the CNT transistor may stem from charge trapping in water molecules around the CNT, where the CNT is exposed to the ambient environment.19 The trapping/detrapping effect of carriers provides internal dynamics that drive the analogue-channel conductance-switching behavior, which is described in detail in Supporting Information Note 3. In fact, hysteresis has frequently been observed in CNT transistors and is difficult to eliminate. As reported in other studies,20,21 the modification of transistor structures and use of hydrophobic film may reduce the hysteresis, and therefore, the additional charge trap layer might be necessary to allow electrons or holes to be trapped or detrapped reliably. However, we intentionally utilized the back-gate transistor structure in this work, which is exposed to the ambient environment, and large hysteresis was obtained for neuromorphic network operation. To identify the analogue conductance-switching behavior in a synaptic transistor, Figure 2b shows the schematics of the pulse trains used for the measurement. Each pulse train consists of 75 pulses (+5 V for potentiation and -5 V for depression) followed by non-perturbative read voltage pulses at -3 V within the intervals. Note that the measured channel conductance (G) during the applied pulse trains show a gradual transition in both potentiation and depression responses. In particular, the range of the analogue conductance


7


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 8 of 28

modulation (∆G = maximum G - minimum G) is approximately 100%, which is clearly distinct from the previous memristor-based synaptic devices, whose modulation range of conductance is at most 30%.4,22 A larger ∆G margin enables more efficient learning and pattern recognition in neuromorphic network operation, which is discussed later; this is a notable merit of the demonstrated CNT synaptic transistor compared to previous memristors. Next, the synaptic learning rule (STDP) refers to the effect that the relative timing of pre- and post-synaptic spikes determines the sign and magnitude of the long-term synaptic weight change. The long-term modification of channel conductance in a CNT synaptic transistor should be reproducible by a pair of temporally correlated gate (VG) and source (VS) voltages. To systematically study how the channel conductance is affected by VG and VS, the potentiation and depression responses of the channel conductance in a CNT synaptic transistor were measured at different VG/VS pulse configurations, as shown in Figure 3a. By keeping the gate pulse amplitude constant at +6 V and -6 V for potentiation and depression, respectively, while changing the source pulse amplitude from 0 to -1 V during repeated potentiation and depression processes. Figure 3b shows the evolution of the channel conductance as a function of the net voltage difference between the gate and source pulses (VGS). As shown in Figure 3b, distinctive analogue conductance-switching behaviors were observed to be dependent on VGS; a higher |VGS| was found to yield a larger conductance change. When VS = -1 V during the potentiation process, i.e., VGS = +7 V, a larger ∆G was obtained compared with the case of VGS = +6 V (i.e., VS = 0 V). Similarly, when VS = -1 V during the depression process, i.e., VGS = -5 V, a smaller ∆G was obtained compared with the case of VGS = -6 V (i.e., VS = 0 V). This phenomenon occurred because a higher VGS causes more trapping of carriers during the potentiation process, which results in a larger increase in conductance. Similarly, a higher |VGS| during the depression


8

Page 9 of 28

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


process enhances the detrapping process and thus leads to a larger decrease in conductance. As a result, the channel conductance of a CNT synaptic transistor can be controlled based on the net voltage difference between the gate and source terminals, which enables the emulation of the STDP effect based on the temporally correlated pre- and post-synaptic spikes. In particular, the ability to control the channel conductance from the correlated VG and VS voltages is clearly distinct from previous CNT synaptic transistors23,24; because channel conductance modulation was only possible through the gate terminal, either the STDP effect was not reproducible23 or two paired-transistors were used to implement the potentiation/depression processes,24 which are inefficient methods to produce a large-scale neuromorphic system. Due to this distinctive controllability of channel conductance in our CNT synaptic transistors, the learning rule (STDP) was consequently explored using a designed spike paring protocol. The bipolar, saw-edged pulses shown in Figure 3c were used for both pre- and post-synaptic spikes (Vpre- and Vpost-), where Vpre- and Vpost- are applied to the gate and the source terminals, respectively. Then, the net voltage difference (VGS = Vpre- - Vpost-) is applied to the device at the moments of a positive or negative time interval (∆t). The polarity of VGS is determined by the sign of ∆t; VGS is positive when ∆t > 0 for potentiation and negative when ∆t < 0 for depression. Figure 3c shows the measured rate of conductance change as a function of the relative timing of the pre- and postsynaptic spike applications, which is indeed consistent with the biological data measured in hippocampal glutamatergic synapses measured by Bi and Poo (the long-term characteristic of STDP is verified in Supporting Information Note 4).3 Significantly, the change in conductance is shown to depend on the voltage difference between the two terminals, which is a key factor in implementing a CNT synaptic transistor network, as discussed below.


9


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 10 of 28

Here, we present system-level simulation results using a simplified STDP scheme23 and show how the demonstrated CNT synaptic transistors associated with CMOS neuron circuits can perform unsupervised learning and pattern recognition (Figure 4a). The detailed simulation parameters and model used in this study are described in Supporting Information Note 5. In principle, sensors will sense the external environment and then convert the sensed information into the pre-synaptic spikes with timings. When an input neuron fires a pre-synaptic spike (Vpre-), this spike is applied to the gate terminal of a synaptic transistor during tpre (Figures 4b and 4c). Vpre- is sufficient to drive a post-synaptic current in a synaptic transistor but not sufficient to modulate the channel conductance for potentiation/depression. Next, the post-synaptic current is integrated by the output neurons through I&F circuits based on CMOS. Because many synaptic transistors connected to the same output neuron are active concurrently, their post-synaptic currents are summed. Then, if the accumulated post-synaptic current level is above the threshold value, the output neuron fires a post-synaptic spike (Vpost-) back to the source terminal of the synaptic transistor. Thus, a time interval (∆t) between Vpre- and Vpost- exists. As shown in Figure 4c, if no Vpre- pulse is applied to the device (∆t > tpre), then the net voltage difference applied to the gate/source terminals (VGS = Vpre- - Vpost-) is sufficient to initiate the potentiation process, which can increase the channel conductance. Conversely, if the input neuron has fired recently and Vpre- is still being applied to the device, VGS actually decreases the channel conductance. To model the modulation of the channel conductance based on VGS in the proposed system simulations, we use the measured data shown in Figure 2b. An increase in the channel conductance is fitted by the equation:

δG p = γ p + α p e

−β p

G − Gmin Gmax − Gmin

.

(1)


10

Page 11 of 28

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Similarly, a decrease in the channel conductance is described by: δG d = γ d + α d e

−β d

G max − G G max − G min

.

(2)

In this simplified model, the change in channel conductance is only dependent on the present conductance value (G), which is not consistent with the STDP effect shown in Figure 3c. In a real biological system, an enhanced change in the synaptic weight (channel conductance) occurs as the time interval (∆t) between Vpre- and Vpost- is shortened. Therefore, for the purely bioinspired system simulation, the model should consider the effect of ∆t. However, compared with the complex pulse waveform shown in Figure 3c that creates the STDP effect, no bipolar, sawedged pulse is necessary for the Vpre- and Vpost- waveforms in the simplified scheme, which should make the neuron circuitry much easier to design. In addition, the analogue conductanceswitching data shown in Figure 2b are only required for the system simulation, which enables an efficient analysis of a neuromorphic system within a shorter simulation time. Therefore, we use a simplified STDP scheme that is easier to implement. Next, as mentioned earlier, the neuron functions can be emulated using an integrate-and-fire (I&F) circuit that allows neurons to generate spikes and integrate their input, which is meant to solve the simple following equation:

τ

dX + X = I input dt

,

(3)

where we define the state variable (X) to be an integrated post-synaptic current (Iinput), and τ is a leak-time constant. The neuron fires a post-synaptic spike if X reaches a given threshold, Xth. In addition, when an output neuron fires spikes, it sends inhibitory signals to the other output neurons that prevent them from firing during the specific inhibition time and resets their X values


11


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 12 of 28

to zero, which is known as the winner-takes-all rule.26 Simple algorithms for the homeostasis27 effect were included in these simulations. A target activity (i.e., the number of times an output neuron should fire within the total simulation time) is defined for the neurons. Then, the threshold of the neuron (Xth) is increased if the average activity of the neuron is above the target and decreased if it is below the target: dX th = γ (A −T) dt

,

(4)

where A is the mean firing rate of a neuron, T is the target activity, and γ is a constant. Accordingly, all thresholds of the output neurons are adjusted continuously, where Xth increases if the specific neuron fires more than others, and Xth decreases if the specific neuron fires less than others. To demonstrate pattern recognition in a CNT synaptic transistor network, we use the widely studied case of handwritten number recognition using the MNIST database, which consists of handwritten number that are 28 × 28 pixels from 250 writers.28 To guide the learning process, we input the full MNIST training database, which consists of 60,000 digits, into the system. Each input neuron is connected with one pixel of the image; thus, a total of 28 × 28 input neurons emit pre-synaptic spikes that are proportional to the pixel intensity, as shown in Figure 4a. Input presynaptic spikes generate post-synaptic currents based on the synaptic weight of each synaptic transistor and are integrated by output neurons. Then, the one neuron whose integrated postsynaptic current is the highest fires post-synaptic spikes (Figure 4b); correlated pre- and post- resynaptic spikes result in channel-conductance potentiation or depression (i.e., the learning process). After completing the learning process, the network is then tested on the MNIST test database, which consists of 10,000 digits that were not available during training. Figure 5a plots


12

Page 13 of 28

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


the channel conductance learned by the synaptic transistor network in a configuration with ten output neurons (N1 to N10). It is shown that without any supervision and using only a simplified STDP rule, the network learns and memorizes 10 different numbers. To evaluate the capability of the system, we define the recognition rate (RR); for example, as shown in Figure 5b, when the digit ‘1’ is input to the system during the testing process, the N4 neuron primarily fires, which indicates that the integrated post-synaptic current through the N4 neuron is larger than that of the other neurons. This occurs because input pre-synaptic spikes that convert the pixel intensity information of the digit ‘1’ are well matched to the trained synaptic weight of the synapses connected with the N4 neuron, which leads to the largest post-synaptic currents. However, the digit ‘1’ also can cause other neurons to fire occasionally (i.e., a recognition error). This error is worse in the case of the digit ‘9,’ as shown in Figure 5b, because the digit ‘9’ can be confused with the digits ‘4’ or ‘7.’ To quantitatively evaluate the network’s recognition rate, Figure 5c plots the average recognition rate of the ten numbers. With ten output neurons, the recognition rate reaches 60% after 60,000 training instances and can be increased as the number of training is increased. Additionally, to provide a guideline on how to improve the recognition rate, we investigate the influence of the conductance-switching margin in synaptic transistors on the recognition rate of the network. For this study, we arbitrarily design two different analogue conductance-switching behaviors, as shown in Figure 5d. Case 1 has a larger range of conductance modulation (∆G) compared with the measured data shown in Figure 2b. Conversely, case 2 has a smaller ∆G so that the recognition rate, which is dependent on ∆G, can be investigated by simulation. Figure 5e shows the simulation results with different ∆G cases and shows that the recognition rate is significantly affected by ∆G, where a larger ∆G produces a better recognition rate. This result indicates that ∆G is an important parameter in synaptic


13


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 14 of 28

devices. The demonstrated CNT synaptic transistor is shown to have a larger ∆G margin than previous memristor-based synaptic devices, which enables more efficient pattern recognition in a neuromorphic network.

CONCLUSIONS In summary, in this study, we have experimentally demonstrated an electronic synapse device based on CNT transistors with reliable, analogue, conductance-modulation behavior. Specifically, the internal dynamics of a CNT transistor enables a native mechanism to encode the synapticweight plasticity through the gate/source terminals. These findings facilitate the potential development of a large-scale CNT synaptic transistor network without having to consider deviceto-device variability. Additionally, a simplified STDP scheme was used to simulate the network at a system level, where CNT synaptic transistors associated with CMOS neuromorphic circuits could perform unsupervised learning and pattern recognition. The range of conductance modulation in CNT transistors is significantly larger than that of typical memristors, which enables a better recognition rate during pattern recognition operations. This simulation result is an important step towards the effective analogue hardware implementation of more complex neuromorphic networks Future work should focus on the experimental demonstration of these concepts beyond a single device and a large-scale synaptic transistor network by using a scaled CNT transistor; a demonstration of the short-channel CNT transistor with a sufficient analogue conductance-modulation margin is required. Based on previous studies,29,30 we believe that the use of a higher-purity semiconducting enriched CNT solution and/or a single semiconducting CNT in the device channel can provide a sufficient conductance-modulation margin even in scaled, short-channel CNT transistors. Although we focused on the proof of concept in this study,


14

Page 15 of 28

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


i.e., the feasible application of CNT synaptic transistors, we expect that the platform and concept presented here will be beneficial for the development of high-density neuromorphic systems.

ASSOCIATED CONTENT Supporting Information This supporting information provides additional discussions regarding the electrical properties of CNT synaptic transistors and their variability, details on the hysteresis mechanism in CNT synaptic transistors, and details on the pattern recognition simulation parameters and long-term effect of STDP. This material is available free of charge via the Internet at http://pubs.acs.org.

Author Information Correspondence

and

requests

for

materials

should

be

addressed

to

S.

J.

C.

([email protected]). S. Kim, J. Yoon, and H. –D. Kim are equally contributed this work.

Acknowledgements The work was supported by the National Research Foundation of Korea through the Ministry of

Education,

Science

and

Technology,

Korean

Government,

under

grant

no.

2013R1A1A1057870.


15


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 16 of 28

REFERENCES (1) Jain, A. K.; Mao, J.; Mohiuddin, K. M. Artificial Neural Networks: A Tutorial. Computer 1996, 29, 31-44. (2) Mead, C. Neuromorphic Electronic Systems. Proc. IEEE 1990, 78, 1629-1636. (3) Bi, G. Q.; Poo, M. M. Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type. J. Neurosci. 1998, 2, 10464–10472. (4) Jo, S. H.; Chang, T.; Ebong, I.; Bhadviya, B. B.; Mazumder, P.; Lu, W. Nanoscale Memristor Device as Synapse in Neuromorphic Systems. Nano Lett. 2010, 10, 1297-1301. (5) Yu, S.; Wu, Y.; Jeyasingh, R.; Kuzum, D.; Wong, H. –S. P. An Electronic Synapse Device Based on Metal Oxide Resistive Switching Memory for Neuromorphic Computation. IEEE Trans. Electron Devices 2011, 58, 2729-2737. (6) Alibart, F.; Pleutin, S.; Bichler, O.; Gamrat, C.; Serrano-Gotarredona, T.; Linares-Barranco, B.; Vuillaume, D. A Memristive Nanoparticle/Organic Hybrid Synapstor for Neuroinspired Computing. Adv. Funct. Mater. 2012, 22, 609–616. (7) Krzysteczko, P.; Munchenberger, J.; Schafers, M.; Reiss, G.; Thomas, A. The Memristive Magnetic Tunnel Junction as a Nanoscopic Synapse-Neuron System. Adv. Mater. 2012, 24, 762– 766.


16

Page 17 of 28

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


(8) Wang, Z. Q.; Xu, H. Y.; Li, X. H.; Yu. H.; Liu, Y. C.; Zhu, X. J. Synaptic Learning and Memory Functions Achieved Using Oxygen Ion Migration/Diffusion in an Amorphous InGaZnO Memristor. Adv. Funct. Mater. 2012, 22, 2759–2765. (9) Zamarreno-Ramos, C. Camunas-Mesa, L. A.; Perez-Carrasco, J. A.; Masquelier, T.; SerranoGotarredona, T.; Linares-Barranco, B. On Spike-Timing-Dependent-Plasticity, Memristive Devices, and Building a Self-Learning Visual Cortex. Front. Neurosci. 2011, 5, 26. (10) Masquelier, T.; Thorpe, S. J. Unsupervised Learning of Visual Features through Spike Timing Dependent Plasticity. PLOS Comput. Biol. 2007, 3, e31. (11) Nessler, B.; Pfeiffer, M.; Buesing, L.; Maass, W. Bayesian Computation Emerges in Generic Cortical Microcircuits through Spike-Timing-Dependent Plasticity. PLOS Comput. Biol. 2013, 9, e1003037. (12) Kim, K. H.; Gaba, S.; Wheeler, D.; Cruz-Albrecht, J. M.; Hussain, T.; Srinivasa, N.; Lu, W. A Functional Hybrid Memristor Crossbar-Array/CMOS System for Data Storage and Neuromorphic Applications. Nano Lett. 2011, 12, 389–395. (13) Wong, H. S. P.; Lee, H. –Y.; Yu, S.; Chen, Y. –S.; Wu, Y.; Chen, P. –S.; Lee, B.; Chen, F. T.; Tsai, M. –J. Metal-Oxide RRAM. Proc. IEEE 2012, 100, 1951-1970. (14) Alibart, F.; Zamanidoost, E.; Strukov, D. B. Pattern Classification by Memristive Crossbar Circuits with Ex-Situ and In-Situ Training. Nat. Commun. 2013, 4, 2072. (15) Eryilmaz, S. B.; Kuzum, D.; Jeyasingh, R.; Kim S.B.; BrightSky, M.; Lam, C.; Wong, H. S. P. Brain-Like Associative Learning Using a Nanoscale Non-Volatile Phase Change Synaptic Device Array. Front. Neurosci. 2014, 8, 205.


17


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 18 of 28

(16) Prezioso, M.; Merrikh-Bayat, F.; Hoskins, B. D.; Adam, G. C.; Likharev, K. K. Strukov, D. B. Training and Operation of an Integrated Neuromorphic Network Based on Metal-Oxide Memristors. Nature 2015, 521, 61-64. (17) Indiveri, G.; Chicca, E.; Douglas, R. A VLSI Array of Low-Power Spiking Neurons and Bistable Synapses with Spike-Timing Dependent Plasticity. IEEE Trans. Neural Netw. Learn. Syst. 2006, 17, 211-221. (18) Liu, S. –C.; Douglas, R. Temporal Coding in a Silicon Network of Integrate-and-Fire Neurons. IEEE Trans. Neural Netw. Learn. Syst. 2004, 15, 1305-1314. (19) Kim, W.; Javey, A.; Vermesh, O.; Wang, Q.; Li, Y.; Dai, J. Hysteresis Caused by Water Molecules in Carbon Nanotube Field-Effect Transistors. Nano Lett. 2003, 3, 193-198. (20) Yang, M. H.; Teo, J. B. J.; Gangloff, L.; Milne, W. I.; Hasko, D. G.; Robert, Y.; Legagneux, P. Advantages of Top-Gate, High-k Dielectric Carbon Nanotube Field-Effect Transistors, Appl. Phys. Lett. 2006, 88, 113507. (21) Franklin A. D.; Koswatta, S. O.; Farmer, D. B.; Smith J. T.; Gignac, L.; Breslin, C. M.; Han, S. –J.; Tulevski, G. S.; Miyazoe, H.; Haensch, W.; Tersoff, J. Carbon Nanotube Complementary Wrap-Gate Transistors, Nano Lett. 2013, 13, 2490-2495. (22) Kim, S.; Choi, S.; Lee, J.; Lu, W. D. Tuning Resistive Switching Characteristics of Tantalum Oxide Memristors through Si Doping. ACS Nano 2014, 8, 10262-10269. (23) Shen, A. M.; Chen, C. –L.; Kim, K.; Cho, B.; Tudor, A.; Chen, Y. Analog Neuromorphic Module Based on Carbon Nanotube Synapses. ACS Nano 2013, 7, 6117-6122. (24) Kim, K.; Chen, C. –L.; Truong, Q.; Shen, A. M.; Chen, Y. A Carbon Nanotube Synapse with Dynamic Logic and Learning. Adv. Mater. 2013, 25, 1693-1698.


18

Page 19 of 28

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


(25) Querlioz, D.; Bichler, O.; Dollfus, P.; Gamrat C. Immunity to Device Variation in a Spiking Neural Network with Memristive nanodevices. IEEE Trans. Nanotechnol. 2013, 12, 288-295. (26) Maass, W. On the Computational Power of Winner-Takes-All. Neural Computation 2000, 12, 2519-2535. (27) Marder, E.; Goaillard, J. –M. Variability, Compensation, and Homeostasis in Neuron and Network Function. Nat. Rev. Neurosci. 2006, 7, 563-574. (28) Lecun, Y.; Bottou, L.; Bengio, Y.; Haffner, P. Gradient-Based Learning Applied to Document Recognition. Proc. IEEE 1998, 86, 2278-2324. (29) Choi, S. –J.; Bennett, P.; Takeo, K.; Wang, C.; Lo, C. C.; Javey, A.; Bokor, J. ShortChannel Transistors Constructed with Solution-Processed Carbon Nanotubes, ACS Nano 2013, 7, 798-803. (30) Choi, S. –J.; Bennett, P.; Lee, D.; Bokor, J. Highly Uniform Carbon Nanotube Nanomesh Network Transistors, Nano Research 2015, 8, 1320-1326.


19


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 20 of 28

Figure Legends Figure 1. CNT synaptic transistor. (a) Schematic illustration showing a synapse connecting a pair of neurons, where the synaptic functions can be emulated by synaptic transistors. (b) A basic neuromorphic unit that comprises several synapses and a neuron. This unit mimics a biological neuron, where the synapse receives spikes from pre-neurons and converts them into currents based on their synaptic strength. The post-neuron performs a spatio-temporal integration of the spikes and generates output spikes (or action potentials). (c) Device schematic for the demonstrated CNT synaptic transistor and an atomic force microscopy image (5 μm × 2.5 μm, zscale is 10 nm) of the single-walled CNT matrix deposited onto Si/SiO2 substrates.

Figure 2. Analogue conductance modulation behavior. (a) Hysteresis of the source current (IS) as a function of the gate voltage (VG) with a constant drain voltage (VD = -1 V). The hysteresis loop direction is anti-clockwise, which indicates that a positive VG increases the channel conductance and a negative VG decreases the conductance. The channel conductance was measured at a specific read voltage (Vread = -3 V) because this Vread is insufficient to modulate the channel conductance. (b) Schematics of the applied pulse trains used to measure the analogue switching behavior. Each pulse train consists of 75 potentiation or depression pulses (5 and -5 V for 10 ms) followed by small, non-perturbative read voltage pulses (-3 V for 100 ms) within the intervals.


20

Page 21 of 28

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


The conductance was measured during the read pulse and plotted as a function of the applied pulse number.

Figure 3. Plots showing the ability to control the channel conductance with the gate/source terminals. (a) Schematics of the applied pulse trains used for the measurement. Each pulse train consists of 75 potentiation or depression pulses applied to the gate (6 and -6 V for 10 ms) along with a different source voltage pulse (0 or -1 V). (b) Measured analogue conductance-switching behaviors based on different VGS. (c) Implementing STDP using a CNT synaptic transistor. The pre-spike voltage (Vpre-) and post-spike voltage (Vpost-) are applied to the gate and source of the synaptic transistor, respectively. The net programming voltage (Vpre- - Vpost-) applied across the device depends on the positive or negative moments Δt. The symbols in the figure indicate experimental data, and the lines are guides for the reader.

Figure 4. Pattern recognition simulation. (a) Learning and testing operation sequences of the CNT synaptic transistor network. The MNIST database consisting of handwritten numbers of 28 × 28 pixels was used to verify the pattern recognition potential of this method. The full MNIST training database (60,000 digits) was input to the system, and input pre-synaptic spikes generate post-synaptic currents, which are then integrated by the output neurons. Then, one neuron fires post-synaptic spikes, and thus, correlated pre- and post- re-synaptic spikes result in channelconductance potentiation or depression (i.e., the learning process). After finishing the learning process, the network is then tested on the MNIST test database, which consists of 10,000 digits that were not available during training. (b) Architecture of the CNT synaptic transistor network, consisting of 28×28 input neurons and 10 output neurons. Lateral inhibition between the output


21


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 22 of 28

neurons is also implemented. (c) Pulses for simplified STDP (voltage pulses as functions of time). The net voltage difference (VGS = Vpre- - Vpost-) is applied to the device at the moments of a positive or negative time interval (Δt). The polarity of VGS is determined by Δt; VGS is positive when Δt > tpre for potentiation and negative when Δt < tpre for depression.

Figure 5. Simulation results. (a) Channel conductance (synaptic weights) learned in a simulation with ten output neurons (N1 to N10). Red indicates a minimum conductance, and blue indicates a maximum conductance. (b) The number of firings that each neuron creates in the case of the digit ‘1’ and the digit ‘9’ after 60,000 training instances. (c) Recognition rate from the test dataset as a function of the training number. Each simulation was repeated five times, and the error bars describe the range of values observed from their minimum to their maximum values. (d) Measured and arbitrarily designed, analogue, conductance-switching behaviors to investigate the influence of the conductance modulation range on the recognition rate of the network. (e) Simulation results showing the recognition rate based on the different analogue conductanceswitching behaviors.


22

Page 23 of 28

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Figure 1


23


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 24 of 28

Figure 2


24

Page 25 of 28

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Figure 3


25


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 26 of 28

Figure 4


26

Page 27 of 28

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Figure 5


27


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 28 of 28

TOC figure


28

Carbon Nanotube Synaptic Transistor Network for Pattern Recognition

Recommend Documents