In the Classroom
On Solving “Problems” Richard Ghez Department of Materials Engineering, The Technion, Haifa 32000, Israel;
[email protected] Who, in class, has not come with relief to the end of a lengthy calculation? The students are also relieved because, glaring down at them from the blackboard, there crouches some ultimate expression, now still and perhaps suitably boxed. Turning towards them, I then ask, “Have we solved our problem?” Their response most often goes from a vacant gaze to incoherent mumbling. Just what is meant by the phrase “finding the solution to a problem”? Of course, we have in mind the state of affairs in the natural sciences, and not in politics or psychology. More precisely, we mean the resolution of a mathematical question that represents, more or less, some other question that arises in the natural sciences. And, unless one is concerned with purely logical steps (i.e., proofs of theorems), we mean finding and evaluating expressions numerically.1 Indeed, most scientists seem fascinated by calculational steps, for instance, see Feynman’s hilarious accounts,2 Mermin’s musings (1), or Sacks’s3 and Hoffmann’s4 secret longings. Because of their outstanding literary skills, both Sacks, the would-be chemist, and Hoffmann, the Nobel laureate in chemistry, help us grasp the unity of all natural sciences and that counting, estimating, and the recognition of patterns are the “glue” that binds them. Examples are better than dull generalities. Beginning with a simple one, everybody “knows” that (1) 2 + 3 = 5 which an ancient Roman might have written5 (2)
II + III = V
One huge advantage, among others, of the number system, eq 1, over the second, eq 2, is that the first requires only 10 symbols to represent any number. However, it is rarely remembered that eq 1 is merely a shorthand notation for
(1
+ 1) + (1 + 1 + 1) = (1 + 1 + 1 + 1 + 1) (3)
and that its binary representation6 is (4)
10 + 11 = 101
Here, it is important to realize that eqs 3 and 4 embrace procedures for finding answers to the questions embodied in the left-hand sides of eqs 1 and 2. In fact, almost every “=” sign indicates a latent question. The first, eq 3, is what we do when “counting on our fingers” and the second, eq 4, is what computers tend to do. To take a slightly less serious question, life insurance companies thrive on a population’s death rate. It is thus vitally important for them to estimate our life expectancy. The physical question is “How long do we live, on average,” assuming that we can agree on what that phrase may mean. Let us suppose that some statistician, working for the XYZ Mutual Insurance Co., estimates our life expectancy to be 70 years and that, for some odd reason, the managers want 610
Journal of Chemical Education
•
it expressed in seconds. A short calculation will yield Your life expectancy is 2,207,520,000 s.
(5)
This is ridiculous, of course, and we may be content with the approximation Your life expectancy is approximately 2.2 × 109 s. (6) which we may reasonably call an “order of magnitude”.7 Then, perhaps using some biological model of cell birth and death rates, a rival statistician (younger and better educated, naturally) comes up with another answer to that same physical question: Your life expectancy is 231 s.
(7)
Who is right? Can we even compare the two results, eq 5 and eq 7? After all, they are expressed in very different notation. Remember that powers are really just another shorthand for some procedure. Here, 2 31 = 2 × 2 × … × 2 (8) 31 factors
an operation that can be carried out, in principle, using purely arithmetic rules. It thus has a decimal representation that we may not care to write out, any more than the result of eq 5. What we want, really, is an approximation of 231 that can be compared to eq 6. That is easy enough because we probably all know that 23 = 8, and even that 210 = 1024 (one “kilobyte”). Thus,
( )
2 31 = 2 × 230 = 2 × 210
3
≅ 2 × 10 9
(9)
and we come to the conclusion that our two statisticians agree, within reason.8 In fact, we can do slightly better if we are willing to leave the domain of pure integers. Since 1024 = (1 + 0.024) × 103, we use Newton’s binomial theorem (1 + x)n = 1 + nx + O(x2), where O represents the higher order terms, to finally get Your life expectancy is approximately 2.14 × 109 s. (10) The managers can now decide whether or not to fire one or both of the statisticians. After all, in the West, our life expectancy is closer to 80 years than to 70. Moral of this tale: check your basic physical assumptions. Let us go on to really large numbers. Anyone will tell you that 5! = 120 in a matter of seconds, and we can all calculate 10! = 3,628,800 within a minute or two. How about 100!? At the rate of one elementary operation per second, it would take roughly 5.5 hours, without stop, to calculate that number by hand.9 But, Stirling’s asymptotic formula n! ∼
2 πn (n e )
n
(11)
to the rescue, together with the useful estimate e3 ≅ 20, and we get 100! ≅ 10158 = (one googol)1.58, by hand, within a
Vol. 83 No. 4 April 2006
•
www.JCE.DivCHED.org
In the Classroom
minute or less. It is fortunate that Boltzmann’s formula for configurational entropy involves ln(n!), only. We now leave integer arithmetic. You have some physical question that can be answered provided you find the roots of
x 2 + 5764 x − 1 = 0
(12)
Aha! a quadratic equation! to which you hasten to apply “the formula”. But that, again, is ridiculous because the coefficient b is quite large,10 relative to a and c, and adding ᎑4ac won’t make much difference to b 2. These are numerical limitations that point to the finite world we live in. Even the largest computer has a finite maximum “word size”. So, let’s use our brains, instead. If you stare at the quadratic in eq 12 for a moment, you will quickly realize that it has two real roots: one very small and positive root, and another that is very, very negative. Indeed, using one of Viète’s formulas, if you have found one root x1, then the other is simply ᎑1兾x1. Can we estimate x1? Why of course, because it is almost immediately obvious that the three terms of eq 12 cannot all be of the same order of magnitude when x ≅ x1. It follows that the small positive root is approximately the solution of the linear equation (13)
5764 x − 1 = 0
which should cause no difficulty, numerical or otherwise. Another example comes to us from ancient Babylon. Close to 4000 years ago, Babylonian mathematicians had found ways of calculating square roots to any accuracy.11 In modern notation, if we want the square root of a positive number c, then calculate iteratively
xn + 1
1 c = xn + ; n = 0, 1, 2, … xn 2
(14)
Of course we must start somewhere, and therefore the “initial condition” x0 is our own choice. The method works surprisingly well.12 A typical mathematician would say that this method is “robust” and “converges” quickly. Try it! While on the topic of finding roots, square or otherwise, of some equation f (x) = 0, it may be safe to say that the landscape of natural sciences would be rather blighted without efficient, iterative, numerical techniques to solve nonlinear equations. Of particular interest to readers of this Journal, any discussion of the law of mass action, and therefore the calculation of phase diagrams, draws heavily on such techniques because the conditions for phase equilibria are generally nonlinear. We are finally in a position to answer our initial question, “What is a solution?” But first, we ask what are the differences and similarities between this last example and the first four. Here, we are required to perform “an infinite number of steps to get the answer √c ”, whatever that may mean. The previous cases were “finite” because only a finite number of arithmetic operations was ever required. All these cases illustrate specific procedures, however. Even the last one is a procedure or algorithm in the sense that each successive approximation (eq 14) consists of a definite (i.e., an unambiguous) and finite collection of arithmetic and logical steps. We stop when we are satisfied with a certain level of accuracy. In short, a (numerical) solution is an algorithm. www.JCE.DivCHED.org
•
Speaking of students’ beliefs and misconceptions, and as I had hinted in the beginning, most students are unhappy if their answers are not in “closed form”, by which they mean some analytic expression.13 Although they are perfectly willing to accept an answer in the form of an integral or a series, they often do not realize that these expressions must be evaluated, as will be emphasized in the last two examples. Moreover, like most ancient Greek mathematicians, they are generally horrified by an endless procedure, such as the Babylonians’ eq 14. For example, the numerical solution of the diffusion equation, ∂C兾∂t = D ∂2C兾∂x2, where D is the diffusion coefficient, can be obtained through the explicit, finite-difference scheme
(
Cin + 1 = (1 − 2 λ )C in + λ C in+ 1 + Cin− 1
)
(15)
Here, the partial differential equation has been discretized, Cin = (xi, tn), and λ is a dimensionless product of D by a ratio of time-to-space mesh sizes (2–3). Mathematicians prove that, as these mesh sizes shrink to zero while keeping λ constant in the interval (0, 1兾2), this algorithm (eq 15) produces an answer to some question in diffusion theory. Moreover, provided that the initial and boundary conditions have been specified,14 we obtain a unique set of numbers, namely, a stable answer for each value of our choice of λ. That, too, is a solution, even as it may bewilder some students.15 To be meaningful, however, any iterative algorithm must be stable. For example, consider the integrals In =
1 n −x 0
x e dx ; n = 0, 1, 2, …
(16)
that are closely related to the gamma function.16 They obviously form a positive and monotonically decreasing sequence. An integration by parts yields the recursion 1 I n = nIn − 1 − (17) e and the “initial condition” I0 = 1 − 1兾e is obtained, directly, from the definition (eq 16). Are we satisfied? The process does appear well behaved with modern calculating devices, as is illustrated in the second column of Table 1. This is an
Table 1. Numerical Evaluation of the Recursion, Equation 17, Showing the Influence of Decimal Accuracy n
“Exact”
8 Decimals
2 Decimals
0
0.6321206
0.6321206
0.63
1
0.2642411
0.2642411
0.26
2
0.1606028
0.1606028
0.15
3
0.1139289
0.1139289
4
0.0878363
0.0878364
0.08 ᎑0.05
5
0.0713022
0.0713023
᎑0.62
6
0.0599336
0.0599345
᎑4.09
7
0.0516558
0.0516618
᎑29.00
8
0.0453668
0.0454152
᎑232.37
9
0.0404221
0.0408576
᎑2091.70
NOTE: The digits in bold indicate differences from the digits in the exact column.
Vol. 83 No. 4 April 2006
•
Journal of Chemical Education
611
In the Classroom
illusion, however. If we attempt to calculate the successive terms of eq 17 with two- or three-decimal accuracy (a “slide rule”), then all hell breaks loose: the terms go negative (super-exponentially) and, in other cases, might even oscillate wildly. The second column, labeled “exact”, was calculated with the best computer I could find (15 significant digits); however, only seven decimals (rounded) are displayed. Let’s trust them for the moment. For the third column, I used my wife’s antique 8-digit calculator, and the last column simulates a slide rule for the decimal part. Deviations from the “exact” results are indicated in bold. They are telling as they creep leftwards. Even my “exact” calculations will eventually go negative and are then worthless. That is numerical instability. It is chaotic. Finally, to somewhat sharpen my argument regarding misconceptions, suppose that the answer to some physical question requires the evaluation of cos(0.6). We are so accustomed to the properties of trigonometric functions that we fail to realize that computing their values is sometimes nontrivial. Of course, your calculator “gives” you cos(0.6) = 0.82533…, but how does it do that? Well, we all know the Taylor expansion ∞
cos x =
x 2n
∑ ( −1)n (2n ) !
n=0
= 1−
x2 x4 + + … (18) 2! 4!
that converges rather nicely. Therefore, we have cos (0.6) = 1 −
0.36 + O (0.6 4 ) ≅ 0.82 2
(19)
which may be satisfactory. If not, we can always include higher-order terms of the series. In any event, we must remember that the series has been truncated. No human being, dead or alive, could ever calculate exactly a numerical series since the number of arithmetic steps is infinite.17 Giving a series (or an integral, or an infinite product, or a continuous fraction) is not an algorithm; it is a transcendental operation. Only its numerical estimation, obtained by successive truncations (the “partial sums”), consists of a finite number of arithmetic steps. In sum, every numerical solution contains an algorithm, and every stable algorithm is the solution of something (4). Although we live in a finite world and can “solve problems,” I find it ever amazing that we can also imagine infinite processes. Acknowledgments I am deeply grateful to my mentor and friend, Ronit Galapo, for teaching that “problems” are largely a figment of our imagination, but that imagination also opens the door to the unbounded. It also pays to have friends who are mathematicians: To Alexandre Chorin go my heartfelt thanks for a critical reading of this manuscript. Notes 1. The following was largely inspired by the situation described in note 15 and by V. A. Uspensky’s delightful monograph
612
Journal of Chemical Education
•
for high school students, Pascal’s Triangle and Certain Application of Mechanics to Mathematics, Little Mathematics Library, Mir Publishers: Moscow, 1976. 2. Many instances can be enjoyed in R. P. Feynman’s bestsellers: Surely, You Are Joking Mr. Feynman! (W. W. Norton Co.: New York, 1985) and What Do You Care What Other People Think? (W. W. Norton Co.: New York, 1988). Two outstanding examples, illustrating extreme attitudes, are reprinted in his The Pleasure Of Finding Things Out (Perseus Books: Cambridge, MA, 1999), “Algebra for the Practical Man”, pp 5–7, and his conversations with H. Bethe around the Marchant calculating machine, in “Los Alamos from Below”, pp 60–61. 3. In Sacks, O. Uncle Tungsten (Vintage Books: New York, 2001), according to its index, the words “mathematics” and “numbers” occur no less than 21 times in distinct contexts. Sacks’s appreciation for counting and estimating stands out even more vividly in his wondrous story of “The Twins,” in his best-seller: The Man Who Mistook His Wife For A Hat (Touchstone: New York, 1970). 4. In Hoffmann, R. The Same And Not The Same (Columbia University Press: New York, 1995), the word “mathematics” is wholly absent from the index, and Hoffmann’s portrayals of mathematicians and theoretical physicists, including his “minor tirades against reductionism” (his epithet, p 22), sometimes verge on the dismissive. Nonetheless, even a cursory glance at this rich and beautiful book suggests that the author is passionately concerned with patterns. And what is that, if not mathematics? For example, see K. Devlin’s lovely Mathematics: The Science of Patterns; Scientific American Library: New York, 1997 or the much more massive tome edited by F. Le Lionnais, Les Grands Courants de la Pensée Mathématique; Les Cahiers du Sud: Paris, 1948; it has undergone numerous re-editions since then. (An English translation of the 1962 edition, Great Currents of Mathematical Thought, has been reprinted, in two volumes, by Dover Publications, New York, 1971.) 5. This is not a laughing matter if you know the numbering system based on the Hebrew alphabet (borrowed from classical Greece) that is still in use. Of course, neither these modern Semites nor the ancient Greeks and Romans have (or had) any symbols to express addition and equality. 6. The advantage of eq 4 over eqs 3 or 1 is that the binary system requires only two symbols, 0 and 1, let us say, and the “addition and multiplication tables” are then as simple as possible. Indeed, Leibniz was so enthralled by the aptness of the binary system that he wanted to bring it to the attention of the Emperor of China, in the hope of converting him to Christianity. After all, if zero is the Void, then one must be the Almighty, Himself. (After this text was first submitted, I received, as if by accident, a recent issue of Mathematics Magazine 2003, 76, 276–291. It contains an article, wonderfully apropos, by F. J. Swetz, entitled, Leibniz, the Yijing, and the Religious Conversion of the Chinese.) 7. It may not be that ridiculous to realize that our life expectancy is only roughly two billion seconds. 8. I once gave undergraduate students a test that included the question, “Estimate 231.” From one student I got the very reasonable answer: “The population of China,” which earned him a perfect score. 9. Most handheld calculators will object violently to 100!. But, since any number lies between two successive powers of ten, it follows that its number of (decimal) digits is roughly its (base 10) logarithm. We can thus estimate the number of elementary operations (multiplications and additions) that are required to calculate 100! by hand. Taking the logarithm of Stirling’s formula, eq 11,
Vol. 83 No. 4 April 2006
•
www.JCE.DivCHED.org
In the Classroom we must perform essentially 0.43 × 3 × ∑99k=11k(lnk − 1) ≅ 19,700 operations. And that is 5.5 hours at the rate of one operation per second. For comparison, 1000! would require more than 53 days (and nights) of continuous calculation. 10. The coefficient b = 5764 was not chosen at random. As of this writing (March 26, 2004), it is the age of the universe according to the Hebrew Bible. 11. Neugebauer, O. The Exact Sciences in Antiquity; Dover Publications: New York, 1969; pp 50, 52. We do not know how the Babylonians devised this trick. We may guess, however. In modern language, the positive root of c is also the positive solution of x2 − c = 0. Expressing this equation as x = c兾x suggests the recursion xn+1 = c兾xn. But this does not work at all! For any initial guess x0, the sequence of values one obtains will oscillate between c兾x0 and x0, which is a simple example of periodic (but not chaotic) behavior. On the other hand, the recursion, eq 14, does appear to be derived from the arithmetic average of the obvious equation x = x and of x = c兾x. 12. It is, in fact, the Newton–Raphson method in disguise. Students are often fascinated by the gradual approach to the required root and its dependence on the initial guess, especially if the calculations are performed, step-by-step, on a handheld calculator. They then sense what it means to converge, what dry “epsilon–delta” proofs fail to convey. 13. Most students are aware of “existence theorems”. This may be the root of their confusion. After all, they think, if a solution “exists”, then it must have an analytic expression. 14. The diffusion equation merely expresses local mass (or energy) conservation. The real physicochemistry lies in the boundary
www.JCE.DivCHED.org
•
conditions. For example, if the concentration can change at boundaries because of heterogeneous chemical reactions, then its values must often be determined by some root-finding scheme. 15. It is amazing to observe students’ responses to the question “have you solved your problem?”, even after they have taken the trouble to calculate the numerical solution of some differential equation. See note 13, above, for a partial explanation. 16. These integrals are “moments” of the exponential on the interval (0, 1). If the interval of integration is extended to infinity, then In = n!, a most remarkable result. Just look up any table of integrals to find those relatively few integrals that can be expressed in terms of integers or rational numbers, only. 17. Of course, I am excluding from consideration those (very few) series that can be summed to yield a rational number.
Literature Cited 1. Mermin, David N. Amer. J. Phys. 1978, 46, 101–105. Mermin, David N. Amer. J. Phys. 1984, 52, 362–365. Mermin, David N. Amer. J. Phys. 1987, 55, 584–585. All these are reprinted in his Boojums All The Way Through; Cambridge University Press: Cambridge, 1990. 2. Crank, J. The Mathematics of Diffusion, 2nd ed.; Oxford University Press: London, 1975; Chapter 8. 3. Ghez, R. Diffusion Phenomena: Cases and Studies; Kluwer Academic/Plenum Publishers: New York, 2001; Sections 1.4–1.5. 4. Forsythe, G. E. Amer. Math. Monthly 1970, 77, 931–956. Those concerned with numerical computation can profit from this thoughtful paper.
Vol. 83 No. 4 April 2006
•
Journal of Chemical Education
613
