The Journal of
Physical Chemistry VOLUME 99, NUMBER 25, JUNE 22,1995
0 Copyright 1995 by the American Chemical Society
LETTERS Experiments on Pattern Recognition by Chemical Kinetics Jean-Pierre Laplante" and Maria Pemberton Department of Chemistry and Chemical Engineering, Royal Military College of Canada, Kingston, Ontario, Canada K7K 5LO
Allen Hjelmfelt and John Ross Department of Chemistry, Stanford University, Stanford, Califomia 94305 Received: February 28, 1995; In Final Form: May 2, 1995@ Experiments on pattern recognition are performed with a network of eight open, bistable, mass-coupled chemical reactors. A programming rule is used to determine the network connectivity in order to store sets of stationary patterns of reactors with low or high concentrations. Experiments show that these stored patterns can be recalled from similar initial patterns. To our knowledge, this is the first chemical implementation of a type of neural network computing device. The experiments on this small network agree with simulations and support the predictions of the performance of large networks.
In a series of articles'-6 we have described the theoretical implementation of logic functions and both sequential and parallel computations by macroscopic chemical kinetics. In two of these articles1.2we described a spatially distributed chemical network capable of recalling stored patterns from related inputs. The network is based on ideas taken from neural network t h e ~ r y . ~ - In I ~ this letter, we report on experiments with a pattem recognition device of this type. The computation process is carried out by a chemical reaction with multiple steady states in a network of continuous flow stirred tank reactors (CSTRs). We have studied a network of eight CSTRs coupled by mass transport, in this case reciprocal mass pumping. Each reactor can exist in one of two stable steady states, and the set of states of the reactors constitutes a pattem; hence each reactor is a pixel in all of the patterns. With a programming rule, arbitrary sets of patterns can be stored in the network. Our chemical network uses bistable elements whereas typical neural networks use monostable McCullouch-Pitts type neurons." There are many physical, chemical, and biological processes which exhibit bistability. One of the best studied bistable chemical reactions is the iodate-arsenous acid reaction @
Abstract published in Advance ACS Abstracts, June 1, 1995.
0022-3654/95/2099- 10063$09.00/0
run in a CSTR.'2,13 When iodate is in stoichiometric excess, the overall reaction is
210,-
+ SH,AsO, + 2H'
-
I,
+ SH,AsO, + H,O
(1)
At low flow rates, only a high iodine state exists, and vice versa. At intermediate flow rates, autocatalysis in iodide results in bistable states of either low or high iodine concentration. The state of each reactor is determined visually by adding starch to the reactor inflows, the high iodine state being blue and the low iodine state colorless. In our experiments each reactor is fed with an identical flow of reagents, at a flow rate corresponding to a point within the bistability limits (see Figure 1). To network the reactors, the CSTRs are connected by mass transport. Mass transport is implemented by reciprocal mass pumping using two 16-channel peristaltic pumps. Both pumps operate at identical flow rates, thereby ensuring that there is no net mass transfer between reactors. It can be shown that systems of this type possess only steady-state attractors.I6 The network is programmed by setting the pumping strengths between every pair of CSTRs such that the stored patterns are stable steady states of the network. A Hebbian type ruleI7 is used to determine these strengths. Labeling the two possible states of 0 1995 American Chemical Society
Letters
10064 J. Phys. Chem., Vol. 99, No. 25, 1995
0
30
60
90
120
Time (min) Figure 1. Experiment on pattern recall in a network of eight CSTRs.
The following patterns were stored in the network Pattem 1, 1011OOO1; pattern 2, 01100110; pattern 3, 11001010. The connections were set according to eq 2 as follows: CSTR 1 to 4, 5 , 8; 2 to 5 , 6, 7 (3); 3 to 4, 6, 8; 4 to 1, 3, 8 (3); 5 to 1, 2, 7; 6 to 2, 3. 7; 7 to 2 (3), 5 , 6; and 8 to 1, 3, 4 (3),where a (3) indicates a connection of strength 31. With this choice of patterns, the network is fully connected. The reactor assembly has been described el~ewhere,'~,'~ and essentially consists of eight 25 mL CSTRs, magnetically stirred at approximately 400 rpm. The unreacted concentrations of reagents after mixing in the reactors M, [H3AsO& = 2.0 x M, [I-]o = were: [I03-]o = 9.5 x 3.3 x M. The reaction was buffered at pH 2.0 and carried out at room temperature. Under these conditions, the bistability limits were found to occur at ko = 0.004 and 0.011 s-' (ko is defined as the inverse of the residence time in the CSTR). The results above were obtained at ko = 0.0093 s - I which, according to numerical simulations, is the equistability point under our experimental conditions. The coupling constant 2.. here defined as the ratio of a single connection exchange flow rate to the volume of the reactor, was equal to 0.00033 s-l. To make sure that a recalled pattem was indeed a stable state of the system, the flows were typically left on for 1-2 h ('6 exchange times) after the recall of a stored pattern.
e
element i in pattern p as = [0,1], then the connection strength between elements i and j is given by
where ?, is the coupling constant, O[x] = x if x 2 0, and 0 if x -= 0, and the sum runs over the stored patterns. As seen from eq 2 , two elements which are in the same state in the majority of the stored patterns are connected with a strength that depends on the number of stored patterns in which the two elements have the same state. Elements which are in opposite states in the majority of the stored patterns are unconnected. The total number of connections required to store a given set of pattems depends on the number of patterns, but as well on the pattems themselves. With the present experimental setup, a maximum of 16 reciprocal connections are available. With a network of eight CSTRs and 16 connections available, we have found that a wide variety of sets of three patterns could be stored. With three patterns the connection strengths, according to eq 2 , will either be 0, A, or 32. Connections of strength nA are implemented with n connections of strength A. Let us note that even if some of the connection strengths are 0, the network is most likely fully connected, Le., a path exists from each element to every other element in the network, although there may be one or more intervening elements along this path. Let us also add that because of the symmetry of eq 2 , the photographic negative of each of the patterns is implicitly stored as well. A computational experiment is started by first presenting the network with an initial pattern, different from, but bearing some
similarity to one of the stored patterns. This is done by initializing each of the CSTRs in one of the two corresponding stable states. After the initial pattern has stabilized, the mass exchange is turned on and the computational process begins. If the initial pattem is recognized by the network, the CSTRs in the "wrong" state change states and a stored pattem is recalled as the stable steady state of the network. If the initial pattem is not recognized by the network, homogeneity results with each CSTR assuming the same state. Nonideal operation sometimes occurs if stored patterns mix and a pattem with a number of errors relative to one of the stored pattems is recalled. Figure 1 shows the results of one typical experiment on pattem recall. Here the system is presented with an initial condition corresponding to two errors relative to stored pattem 1. CSTR 1 should be in the high iodine state 1 but is initialized in the low iodine state 0, and vice versa for CSTR 2. After the initial pattem has stabilized, the mass exchange between the CSTRs begins, at t = 0 in Figure 1. After 15 min,the network has corrected the error in CSTR 2 ; after 60 min, the error in CSTR 1 has been corrected as well. The correct recall of the stored pattern can easily be understood by looking at the connections. In the example of Figure 1, CSTR 1 is connected to 4, 5, and 8. Since the majority of these three CSTRs are in the high iodine state, they drive CSTR 1 to the high iodine state. Likewise, CSTR 2 is connected to 5, 6, and 7, all of which are in the low iodine state, and this drives CSTR 2 to the low iodine state. It is interesting to note that similar results are obtained in numerical simulations even if the network is chosen to operate using the iodate-arsenous acid reaction under different conditions from the experiments, e.g., excess arsenous acid. This supports the generality of the experimental results. The performance of this chemical network was tested with three different sets of pattems, presenting the network with a variety of initial conditions. The efficiency of the networks in recalling stored patterns was found to be quite good, but to depend on a number of tuning parameters. An important parameter is the position of the system within the bistability limits (Le., reactant flow rates in each CSTR). Experimenta116-'s and analysis show that the relative stability of the bistable states depends on the position within the bistability region. At the equistability point, both states are equally stable and the transition from one state to the other proceeds with equal ease. A CSTR in the low iodine state coupled to a CSTR in the high iodine state is, at the equistability point, as likely to change its state as the reverse situation. Away from this point, it is more difficult to change one of the two stable states to the other. Small networks should therefore function optimally when operated at or near the equistability point. This tendency was indeed observed in our experiments (in the case of the iodatearsenous acid reaction in excess iodate, the equistability point is unfortunately quite close to the upper bistability limit, just below where the high iodine state ceases to exist). For large networks, the position within the bistability region is not so important.'-5 Figure 2 illustrates a result obtained with the set of patterns of Figure 1 but with a reactant flow rate somewhat lower than the equistability point. In this region, the high iodine state is the most stable, and as can be seen, the stable steady state of the network then corresponds to a uniform high iodine state, indicating nonrecognition. It is however interesting to note that the negative photographic image of pattern 1 is recalled as a transient for a substantial period of time (from ~ 3 to0 100 min). This transient recall of stored patterns was also often observed in numerical simulations. Another parameter which seems to play a similar role in affecting the efficiency of the network is the coupling constant,
Letters
J. Phys. Chem., Vol. 99, No. 25, 1995 10065
0
30
60
90
120
150
Time (min) Figure 2. Experiment on pattern recall away from the equistability point. ko = 0.0067 s-l. Patterns and other conditions as per Figure 1.
important role in determining which pattern is recalled. In the case of the patterns of Figure 1, pattem 1 (or its negative photographic image) was recalled most of the time, even when the initial conditions were chosen to be closer to pattern 2 or 3. Pattern 3, on the other hand, could only be obtained if the system was presented with initial conditions corresponding to pattem 3, without any errors. In these experiments, we have chosen the sets of stored patterns so that they require 32 connections or less and so that they are sufficiently different from each other and from uniformity. The properties of these sets of pattems are believed to be typical of random pattems stored in large networks. However, large networks are not as sensitive to the parameters of the system, and as a result, their computational properties are much more robust. It is nevertheless encouraging (and perhaps somewhat puzzling) to find that an eight-element chemical network can, when properly tuned, be an efficient pattem recognition device. These experiments support the generality of the neural networks approach to computation. Concepts from neurobiology such as McCulloch-Pitts neurons and Hebbian rules were taken for use in models of parallel computation. These concepts were inspired by biological networks and have been implemented in electronic9 and electrooptic hardwares.** Our experiments show that these concepts can be extended to a chemical hardware that has corresponding components in biological systems, that is bistability and mass transfer.
Acknowledgment. This work was supported in part by the National Science Foundation (USA) and the Ministry of National Defence of Canada (ARP FIJHDS). References and Notes 0
30
60
90
120
Time (min) Figure 3. Experiment on pattem recall at high exchange rate (coupling constant). ko = 0.00097 s-l . Patterns and other conditions as per Figure 1.
Le., the exchange rate between reactors. It was found that the network could be "tuned" to be more efficient by adjusting this parameter. In the case of the pattems presented in Figure 1 for instance, pattern 1 was always recalled at the higher exchange rates. The lower exchange rates produced either a homogeneous state (even though again here a stored pattem would sometimes be recalled as a transient), or a stored pattem with one error. The operation at higher exchange rate under optimum conditions can sometimes be quite spectacular. An example is shown in Figure 3. In this experiment, all reactors turned colorless within three minutes after the exchange was tumed on, apparently pointing to nonrecognition. This uniform colorless state persisted for approximately 5 min ( ~ residence 2 times) after which pattem 1 was recalled as the stable steady state. Thus although some CSTRs were still in the high iodine state, there was insufficient iodine to result in a blue color. The relative stability of each of the stored pattems also seems to play an
(1) Hjelmfelt, A,; Schneider, F. W.; Ross, J. Science 1993, 260, 335. (2) Hjelmfelt, A.; Ross, J. J. Chem. Phys. 1993, 97, 7988. (3) Hjelmfelt, A,; Weinberger, E. D.; Ross, J. Proc. Natl. Acad. Sci. U.S.A. 1991. 88, 10983. (4) Hjelmfelt, A.; Weinberger, E. D.; Ross, J. Proc. Natl. Acad. Sci. U.S.A. 1992, 89, 383. (5) Hjelmfelt, A.; Ross, J. Proc. Natl. Acad. Sci. U.S.A. 1992,89, 388. (6) Hjelmfelt, A.; Ross, J. Proc. Natl. Acad. Sci. U.S.A. 1994, 91, 63. (7) Little, W. Math. Biosci. 1974, 19, 101. (8) Hopfield, J. J. Proc. Natl. Acad. Sci. U.S.A. 1982, 79, 2554. (9) Hopfield. J. J. Proc. Narl. Acad. Sci. U.S.A. 1984, 81, 3089. (10) Hopfield, J. J.; Tank, D. Science 1986, 233, 625. (11) McCullouch, W.; Pitts, W. BulL Math. Biophys. 1943, 5, 115. (12) De Kepper, P.; Epstein, I. R.; Kustin, K. J. Am. Chem. SOC. 1981, 103, 6121. (13) Pifer, T.; Ganapathisubramanian, N.; Showalter, K. J. Chem. Phys. 1985, 83, 1101. (14) Laplante, J.-P.; Emeux, T. J. Phys. Chem. 1992, 96, 4931. (15) Laplante, J.-P.; Emeux, T. Physica A 1992, 188, 89. (16) Stuchl, I.; Marek, M. J. Chem. Phys. 1982, 77, 2956. (17) Harding, R.; Ross, J. J. Chem. Phys. 1990, 92, 1936. (18) Foerster, P.; Zhang, Y.-X.;Ross, J. J. Phys. Chem. 1993,97,4708. (19) Hunt, K. L. C.; Kottalam, J.; Hatlee, M. D.; Ross, J. J. Chem. Phys. 1992, 96, 7019. (20) Schlogl, F. 2. Phys. 1972, 253, 147. (21) Ross, J.; Hunt, K. L. C.; Hunt, P. J. Chem. Phys. 1988, 88, 2719. (22) Yu, F.; Jutamalia, S. Optical Signal Processing, Computing, and Neural Networks, Wiley: New York, 1992. JF'950561Q
