Developing Knowledge-Based Systems - ACS Symposium Series

Chapter 18

Developing

Knowledge-Based Systems A Learning Process

Sterling A. Tomellini1, Barry J. Wythoff , and Hugh B. Woodruff Downloaded by EAST CAROLINA UNIV on January 15, 2018 | http://pubs.acs.org Publication Date: September 1, 1989 | doi: 10.1021/bk-1989-0408.ch018

1

2

2

1Department of Chemistry, University of New Hampshire, Durham, N H 03824 M e r c k Sharp and Dohme Research Laboratories, Rahway, N J 07065

A great deal of effort and resources are being dedicated to developing knowledge-based systems in Analytical Chemistry. Creating such complex systems is not always a straightforward process. For example, during the approximately eight years which it has taken to develop PAIRS (the Program for the Analysis of Infrared Spectra), many complications have arisen and modifications made which were not originally anticipated. The knowledge gained while developing PAIRS indicates a number of difficult questions must be addressed before, during, and after creation of such a knowledge-based system. For example, questions concerning the source(s) of knowledge, the scientific merit of the project, the anticipated users of the system and their underlying knowledge of the area, the long-term durability and viability of the system, the methods to be used to evaluate the "final" product, and user accessibility of the knowledge base, must at some point, be addressed. The Program for the Analysis of Infrared Spectra, PAIRS, is a knowledge-based system designed to assist chemists in determining which functional groups are likely to be present in an unknown compound based on its infrared (IR) spectrum. PAIRS can be further classified as a rule-based expert system. While PAIRS was certainly not the first such system to be developed, it was one of the pioneering applications of knowledge-based techniques in analytical chemistry. Many advances have occurred in computer resources during the decade since development work first began on PAIRS. Knowledge-based and expert systems have become the objects of much attention, largely due to this progress, which has placed far more powerful, less expensive, and easier to use computational tools in the hands of researchers. Watson and Mann (1) reported that 150 of the Fortune 500 companies collectively have spent an estimated $1 billion on expert system development. They also cited predictions of $5-10 billion in spending by 1990, $30-40 billion by 1995, and $50-110 billion by the year 2000. The actual impact of this activity on the way scientists do research and the way people live their daily lives is only 0097-6156/89/0408-0236S06.00/0 o 1989 American Chemical Society Hohne and Pierce; Expert System Applications in Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 1989.

Downloaded by EAST CAROLINA UNIV on January 15, 2018 | http://pubs.acs.org Publication Date: September 1, 1989 | doi: 10.1021/bk-1989-0408.ch018

18. TOMELLINI ET AL.

Developing Knowledge-Based Systems

237

b e g i n n i n g t o be f e l t . F o r most p e o p l e , t h e q u e s t i o n , " E x c l u d i n g t h e s y s t e m s w h i c h you have d e v e l o p e d , how many k n o w l e d g e - b a s e d s y s t e m s do you use r o u t i n e l y ? " , r e s u l t s i n an answer o f " z e r o " . D u r i n g t h e 1990s, i t i s e x p e c t e d t h e answer t o t h e above q u e s t i o n w i l l l i k e l y change f o r many p e o p l e . With a l l o f t h e development e f f o r t s c u r r e n t l y underway, i t i s u s e f u l t o r e f l e c t on some o f t h e p r o b l e m s e n c o u n t e r e d and the s o l u t i o n s implemented w h i l e c r e a t i n g t h e PAIRS s y s t e m . Starting f r o m t h e p r e c u r s o r s o f PAIRS, Munk's CASE program (2.) and Wipke's SECS program Q ) , and l e a d i n g up t o t h e c u r r e n t v e r s i o n o f PAIRS, it has been a continuous development process and learning experience. F i g u r e 1 summarizes PAIRS p r o g r e s s i o n s and cites relevant references. Some o f t h e i s s u e s t h a t have had t o be a d d r e s s e d d u r i n g t h i s p e r i o d w i l l be d e s c r i b e d i n t h i s chapter. Where a p p r o p r i a t e , t h e s e i s s u e s w i l l be g e n e r a l i z e d i n an a t t e m p t t o p r o v i d e some i n s i g h t f o r o t h e r r e s e a r c h e r s i n t h i s f i e l d . What Are

the E x p e c t a t i o n s

f o r the

System?

O b v i o u s l y , b e f o r e u n d e r t a k i n g any p r o j e c t , knowledge-based systems i n c l u d e d , t h e o b j e c t i v e s s h o u l d be c l e a r l y e s t a b l i s h e d . The intended a u d i e n c e needs t o be d e t e r m i n e d . D e c i s i o n s must be r e a c h e d about whether t h e system w i l l be n a r r o w l y f o c u s e d t o s o l v e a s p e c i f i c p r o b l e m , or w i l l be l a r g e r and more a m b i t i o u s . When d e v e l o p i n g k n o w l e d g e - b a s e d s y s t e m s , t h e s o u r c e ( s ) t o be u s e d t o o b t a i n t h e r e q u i r e d e x p e r t i s e i s a complex i s s u e . Once t h i s e x p e r t i s e has been t r a n s f o r m e d i n t o t h e knowledge b a s e (or r u l e s ) t o be used by t h e system, a d e c i s i o n needs t o be made about whether o r not t h e r u l e s w i l l be a v a i l a b l e and m o d i f i a b l e by t h e u s e r . F i n a l l y , d e c i s i o n s must be made as t o what w i l l c o n s t i t u t e an answer and what i n f o r m a t i o n w i l l u l t i m a t e l y be p r o v i d e d by the system t o the u s e r . I n t e n d e d A u d i e n c e and System M a g n i t u d e . The e x p e c t a t i o n s o f PAIRS f r o m i t s i n c e p t i o n were t o a s s i s t the p r a c t i c i n g chemist in i n t e r p r e t i n g an IR s p e c t r u m . T h i s g o a l was t o be a c c o m p l i s h e d by c r e a t i n g a s y s t e m which m i m i c k e d t h e thought' p r o c e s s u s e d by an experienced spectroscopist. Thus, much o f t h e e f f o r t i n c r e a t i n g PAIRS i n v o l v e d i n c o r p o r a t i n g t h e knowledge and l o g i c u s e d by a spectroscopist into an extensive rule base. Simply making p r e d i c t i o n s about a l i m i t e d number o f c h e m i c a l f u n c t i o n a l i t i e s o r o t h e r w i s e a r t i f i c i a l l y l i m i t i n g t h e domain o f t h e s y s t e m t o a few chemical functionalities would not suffice i n meeting the e x p e c t a t i o n s e s t a b l i s h e d at the o u t s e t . Thus, i n t e r p r e t a t i o n r u l e s were i n i t i a l l y w r i t t e n f o r 175 chemical functionalities and subfunctionalities. A s c h e m a t i c d i a g r a m o f t h e PAIRS s y s t e m i s g i v e n i n F i g u r e 2. To use PAIRS, t h e s c i e n t i s t was r e q u i r e d t o s u p p l y the s y s t e m w i t h peak i n t e n s i t y , w i d t h , and p o s i t i o n i n f o r m a t i o n d e r i v e d from t h e unknown compound's spectrum, as w e l l as i n f o r m a t i o n c o n c e r n i n g t h e sample s t a t e (e.g., s o l v e n t , n e a t , m u l l ) . In a d d i t i o n , t h e s c i e n t i s t was a l l o w e d t o i n d i c a t e t h e p r e s e n c e o r absence o f g i v e n atom t y p e s i n t h e e m p i r i c a l f o r m u l a o f t h e unknown compound. Since t h e g o a l i n p e r f o r m i n g a PAIRS i n t e r p r e t a t i o n was t o p r o v i d e t h e u s e r w i t h a f u n c t i o n a l g r o u p a n a l y s i s , t h e r e s u l t s o f an e a r l y PAIRS i n t e r p r e t a t i o n were l i m i t e d t o a n u m e r i c a l i n d i c a t i o n o f t h e l i k e l i h o o d o f p r e s e n c e o r absence f o r a g i v e n f u n c t i o n a l i t y . An example o f t h e i n t e r p r e t a t i o n r e s u l t s p r o v i d e d by an e a r l y v e r s i o n

Hohne and Pierce; Expert System Applications in Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 1989.

238

EXPERT SYSTEM APPLICATIONS IN CHEMISTRY


HISTORY OF PAIRS

1 9 80

IBM mainframe version of PAIRS

(4)

1981

Minicomputer version of PAIRS

(5)

1 9 84

First attempts at automated rule generation

(6)

1 9 85

Rule trace incorporated (VAX version)

(7)

1 9 86

A version of PAIRS which allows

1 987

interactive

optimization developed

(8)

PAIRS modified for personal computers

(9)

- explains rationale for decisions

- allows interactive interpretations - CAI of the spectral interpretation process possible

Note:

References in parentheses.

Figure

1.

Significant PAIRS.

advances

during

t h e development

SPECTRAL DATA

RULES

\ /

PAIRS INTERPRETER

» RESULTS

SUPPLEMENTAL DATA F i g u r e 2. A s c h e m a t i c d i a g r a m o f t h e PAIRS

system.


of

18. TOMELLINIET AL.



o f PAIRS f o r t h e s p e c t r a l p r e s e n t e d i n F i g u r e 3.

data

f o r the

compound

nitrotoluene

239 is

Source of E x p e r t i s e . Does one always need an e x p e r t t o c r e a t e a knowledge-based system? T h i s q u e s t i o n stems from t h e o f t e n s t a t e d a s s u m p t i o n t h a t t o make a knowledge-based system, one needs t o f i n d an e x p e r t i n t h e c h o s e n f i e l d f r o m whom t o e x t r a c t t h e r e q u i r e d knowledge. W h i l e t h i s may o f t e n be t h e c a s e , i t i s not a l w a y s n e c e s s a r y t o have an e x p e r t a v a i l a b l e t o d e v e l o p a knowledge-based system. Clearly, the needed commodity i s knowledge, not n e c e s s a r i l y i n t h e form o f a p e r s o n . I f t h e r e q u i r e d knowledge i s a v a i l a b l e f r o m o t h e r s o u r c e s , t h e n an e x p e r t may not need t o be p r e s e n t f o r system development. An example o f a knowledge-based s y s t e m c r e a t e d w i t h o u t t h e d i r e c t i o n o f an e x p e r t i s PAIRS. The knowledge and l o g i c u s e d t o interpret IR s p e c t r a are well documented i n the scientific literature. Thus, t h e b a s i s f o r t h e s y s t e m i s r e a d i l y a v a i l a b l e . W h i l e a number o f g i f t e d i n f r a r e d s p e c t r o s c o p i s t s p a r t i c i p a t e d i n t h e development o f t h e PAIRS knowledge base, t h e knowledge base was prepared primarily using literature resources by analytical c h e m i s t s w i t h some b a c k g r o u n d i n s p e c t r a l i n t e r p r e t a t i o n but who were (and a r e ) by no means i n f r a r e d spectroscopists. The i m p o r t a n c e o f t h e a b i l i t y t o c o n s u l t w i t h e x p e r t s i s , however, v e r y u s e f u l and c a n add g r e a t l y t o t h e o v e r a l l p e r f o r m a n c e o f t h e knowledge-based system. Our e x p e r i e n c e w i t h PAIRS d e m o n s t r a t e d that experts are e s p e c i a l l y helpful i n d i a g n o s i n g where t h e p r e l i m i n a r y system f a i l e d . The a v a i l a b i l i t y o f s u c h i n f o r m a t i o n a i d e d g r e a t l y i n t h e r e f i n e m e n t o f t h e PAIRS system. I f one c h o o s e s t o u t i l i z e an e x p e r t d u r i n g t h e development o f a knowledge-based system, t h e n one must be c o n s c i o u s o f t h e b i a s e s which t h a t e x p e r t has. F o r example, i f one were t o make a knowledge-based s y s t e m t o a s s i s t w i t h c h r o m a t o g r a p h i c s e p a r a t i o n s u s i n g an e x p e r t chromatographer t o p r e p a r e t h e knowledge base, t h e n i t i s r e a s o n a b l e t o expect t h a t the system produced w i l l c o n t a i n the b i a s e s of the chromatographer. Chromatographers, f o r obvious reasons, are g e n e r a l l y forced to limit their thinking to chromatographic experiments which can be performed i n the laboratory. The knowledge-based system, however, w i l l n e c e s s i t a t e t h a t a somewhat p o w e r f u l computer w i l l be a v a i l a b l e t o t h e p e r s o n p e r f o r m i n g the chromatographic experiments. It i s possible that t h e way one approaches the chromatographic problem having a computer a v a i l a b l e may be d i f f e r e n t t h a n t h e way one would a p p r o a c h t h e p r o b l e m w i t h o u t a computer, j u s t as t h e way one approaches a pH t i t r a t i o n depends on whether one must use v i s u a l i n d i c a t o r s o r a pH meter. C l e a r l y , t h e a v a i l a b i l i t y o f t h e computer c h a n g e s t h e resources available to the chromatographer and p o s s i b l y the approaches taken to s o l v e the chromatographic problem. The knowledge and l o g i c u s e d by t h e " e x p e r t " i n c h r o m a t o g r a p h y may, t h e r e f o r e , not be a p p r o p r i a t e as t h e b a s i s f o r a knowledge-based system. Further, i f t h e knowledge and logic u s e d by the chromatographer i s i n c o r p o r a t e d i n t o the knowledge-based system, t h e n t h e s y s t e m w i l l have t h e same l i m i t a t i o n s and b i a s e s as t h e chromatographer. A v a i l a b i l i t y and M o d i f i a b i l i t y o f t h e Knowledge Base. A decision was made e a r l y i n t h e development o f PAIRS t h a t t h e i n t e r p r e t a t i o n r u l e s s h o u l d be made a v a i l a b l e t o t h e u s e r i n a r e l a t i v e l y e a s y - t o u n d e r s t a n d f o r m a t . The r e a s o n s f o r making t h i s d e c i s i o n i n c l u d e d :



240


1) .

To a l l o w t h e u s e r t o d e t e r m i n e t h e d e c i s i o n s which were made to a r r i v e at the r e s u l t s of a given i n t e r p r e t a t i o n ;

2) .

To a l l o w f o r t h e e x p a n s i o n o f t h e r u l e base by t h o s e p e r s o n s w i t h e x p e r t i s e i n i n t e r p r e t i n g s p e c t r a f o r f u n c t i o n a l i t i e s not i n c l u d e d i n t h e PAIRS r u l e base;

3) .

To a l l o w u s e r s who a r e d i s s a t i s f i e d w i t h t h e r e s u l t s p r o v i d e d by t h e i n t e r p r e t a t i o n r u l e s t o m o d i f y , a d j u s t , o r r e p l a c e t h e e x i s t i n g PAIRS r u l e s f o r t h e f u n c t i o n a l i t i e s i n q u e s t i o n .

The i m p o r t a n c e o f a l l o w i n g t h e u s e r t o d e t e r m i n e q u i c k l y t h e r a t i o n a l e used t o a r r i v e a t t h e r e s u l t s o f a g i v e n spectral interpretation cannot be o v e r e s t i m a t e d . Simply making t h e i n t e r p r e t a t i o n r u l e s a v a i l a b l e does n o t a d e q u a t e l y f u l f i l l the needs o f t h e u s e r . In f a c t , i n f o r m a t i o n c o n c e r n i n g t h e r a t i o n a l e f o r t h e d e t e r m i n a t i o n s b e i n g made by t h e system i s o f much g r e a t e r b e n e f i t t o the user f a c e d with q u e s t i o n a b l e r e s u l t s than t h e r u l e base i t s e l f . I t i s f o r t h i s r e a s o n t h a t v e r s i o n s o f PAIRS were d e v e l o p e d which p r o v i d e t h e user w i t h t h e a b i l i t y t o t r a c e t h e d e c i s i o n making p r o c e s s . The s y s t e m was l a t e r expanded t o i n c l u d e e x p l a n a t o r y statements d e s c r i b i n g the r a t i o n a l e f o r the q u e s t i o n s a s k e d and d e c i s i o n s b e i n g made. C l e a r l y , these l a t e r v e r s i o n s of PAIRS go w e l l beyond s i m p l y making t h e r u l e base a v a i l a b l e and add a new d i m e n s i o n t o t h e i n t e r p r e t a t i o n p r o c e s s . The e x p e r t i s e r e q u i r e d , b o t h i n s p e c t r a l i n t e r p r e t a t i o n and d e v e l o p m e n t o f i n t e r p r e t a t i o n r u l e s , has s e v e r e l y l i m i t e d t h e number o f u s e r s w i l l i n g t o e x t e n d t h e PAIRS r u l e b a s e . Further, t h o s e c a p a b l e o f d e v e l o p i n g PAIRS r u l e s a r e o f t e n i n t e r e s t e d i n i n t e r p r e t i n g s p e c t r a o f compounds which a r e p r o p r i e t a r y o r o f g r e a t specificity and, t h e r e f o r e , of l i m i t e d general interest. Developments i n t h e a v a i l a b i l i t y and use o f e x p e r t s y s t e m s h e l l s and t h e l i m i t a t i o n s o f t h e approach used by PAIRS have a l s o r e d u c e d t h e i n t e r e s t i n t h e e x p a n s i o n o f t h e PAIRS r u l e base by s c i e n t i s t s not i n t h e o r i g i n a l PAIRS d e v e l o p m e n t g r o u p . Thus, i t a p p e a r s f u r t h e r e x p a n s i o n o f t h e g e n e r a l l y a v a i l a b l e PAIRS r u l e b a s e t o i n c l u d e t h e e x p e r t i s e o f o t h e r s w i l l p r o c e e d a t a slow pace, i f a t all. I t was o r i g i n a l l y e n v i s i o n e d t h a t t h e a v a i l a b i l i t y o f t h e rules would p r o v i d e u s e r s with a base f o r m o d i f i c a t i o n and improvement t o s u i t t h e i r needs. However, t h e c o m p l e x i t y o f t h e r u l e b a s e and l a c k o f imbedded comments made r u l e m o d i f i c a t i o n difficult. J u s t as t h e meaning o f a key s e n t e n c e i n a p a p e r c a n be changed by s i m p l e r e w o r d i n g by a w e l l meaning e d i t o r , t h e r e s u l t s o f a s p e c t r a l i n t e r p r e t a t i o n can be c o m p l e t e l y changed by s e e m i n g l y modest m o d i f i c a t i o n s t o t h e i n t e r p r e t a t i o n r u l e s . Our e x p e r i e n c e s i n t h i s a r e a have i n d i c a t e d i t i s p r o b a b l y b e t t e r t o a l l o w t h e i n f o r m e d u s e r t o d e v e l o p h i s o r h e r own i n t e r p r e t a t i o n r u l e s t o augment t h e c u r r e n t r u l e b a s e and l i m i t the m o d i f i c a t i o n of e x i s t i n g r u l e s t o those i n t i m a t e l y f a m i l i a r with the o r i g i n a l development o f t h e s p e c i f i c r u l e s i n q u e s t i o n . W h i l e i t may n o t be i n t h e b e s t i n t e r e s t o f system d e v e l o p e r s t o p r o v i d e u s e r a c c e s s t o t h e knowledge base f o r r e a s o n s o f system s e c u r i t y and i n t e g r i t y , t h e a b i l i t y t o u n d e r s t a n d t h e r e a s o n s f o r d e c i s i o n s w h i c h a r e made and c o n c l u s i o n s w h i c h a r e r e a c h e d i s c l e a r l y of importance t o t h e user. I t w i l l be i n t e r e s t i n g t o see how c o m m e r c i a l l y a v a i l a b l e k n o w l e d g e - b a s e d s y s t e m s p r o v i d e such i n f o r m a t i o n t o the users while p r o t e c t i n g the investment of the


18. TOMELLINI ET AL.

241



system developers. This i s s u e i s e s p e c i a l l y important to those s c i e n t i s t s u s i n g such systems i n t h e p r a c t i c e o f t h e i r r e s e a r c h . What I n f o r m a t i o n W i l l C o n s t i t u t e an Answer? The q u e s t i o n o f what i n f o r m a t i o n w i l l be p r o v i d e d by t h e knowledge-based s y s t e m t o t h e u s e r as t h e r e s u l t s o f a c o n s u l t a t i o n s e s s i o n i s v e r y important. In t h e c a s e o f PAIRS, i t was i n i t i a l l y d e c i d e d t h a t t h e q u e s t i o n which the s y s t e m s h o u l d answer was " I s t h i s f u n c t i o n a l i t y l i k e l y t o be p r e s e n t i n t h e unknown compound?". The answer t o s u c h a q u e s t i o n c l e a r l y f a l l s i n t o a "yes, no, maybe" c a t e g o r i z a t i o n . F o r t h i s reason, t h e o r i g i n a l r e s u l t s o f t h e PAIRS s y s t e m were a n u m e r i c a l i n d i c a t i o n of the l i k e l i h o o d of p r e s e n c e f o r a g i v e n f u n c t i o n a l i t y b a s e d on a 0 t o 1 s c a l e . An example o f t h e r e s u l t s o f an i n t e r p r e t a t i o n a r e p r e s e n t e d i n F i g u r e 3. W h i l e u s i n g such a n u m e r i c a l s c a l e answers t h e q u e s t i o n as s t a t e d above, i t s h o u l d be realized t h a t s u c h an answer i s i n s u f f i c i e n t f o r a r e s e a r c h e r f a c e d w i t h a t y p i c a l f u n c t i o n a l group a n a l y s i s problem. F u r t h e r , i t i s c l e a r t h a t an answer i n t h i s form does not t r a n s f e r a l l o f t h e i n f o r m a t i o n which i s c o n t a i n e d i n t h e knowledge base. Realizing that more information could be transferred from the knowledge base t o the user, a natural development was t o p r e s e n t t h e u s e r w i t h a t r a c e o f t h e d e c i s i o n making p r o c e s s i n a d d i t i o n t o a s i m p l e n u m e r i c a l i n d i c a t i o n of presence f o r a p a r t i c u l a r f u n c t i o n a l i t y . Thus, t h e s y s t e m c o u l d now answer t h e q u e s t i o n , "How d i d you d e t e r m i n e t h e l i k e l i h o o d of p r e s e n c e or absence f o r a g i v e n f u n c t i o n a l i t y ? " . F u r t h e r enhancement of t h e i n f o r m a t i o n t r a n s f e r from knowledge base t o u s e r was p o s s i b l e once the u s e r was c a p a b l e o f d e t e r m i n i n g which d e c i s i o n s were made t o a r r i v e a t the g i v e n c o n c l u s i o n s f o r a p a r t i c u l a r f u n c t i o n a l group. C l e a r l y , t h e u s e r c o u l d b e n e f i t from an answer t o the q u e s t i o n o f why a g i v e n q u e r y was made and what t h e d e s i r e d answer m i g h t be. To p r o v i d e t h e u s e r w i t h this i n f o r m a t i o n , i t was n e c e s s a r y t o code i n t o t h e knowledge b a s e t h e reason f o r the queries, d e c i s i o n s , and actions. A partial interpreter trace, i n c l u d i n g d e s c r i p t i v e comments, for the f u n c t i o n a l i t y " n i t r o " from the i n t e r p r e t a t i o n o f t h e s p e c t r a l d a t a f o r n i t r o t o l u e n e i s g i v e n i n F i g u r e 4. W i t h t h i s advance, PAIRS i s now c a p a b l e o f p r o v i d i n g t h e answers t o t h e q u e s t i o n s , "What i s l i k e l y t o be p r e s e n t ? " , "What i n f o r m a t i o n was used t o make t h e s e d e c i s i o n s ? " , and "What was the b a s i s f o r t h e i n f o r m a t i o n used and t h e d e c i s i o n s which were made d u r i n g t h e i n t e r p r e t a t i o n p r o c e s s ? " . C l e a r l y , t h e i n f o r m a t i o n which i s p r e s e n t l y a v a i l a b l e and p r o v i d e d by PAIRS i s q u i t e d i f f e r e n t from the i n f o r m a t i o n which was t h o u g h t t o be n e c e s s a r y and important when PAIRS was i n i t s i n i t i a l planning stage. M a j o r p r o g r a m and knowledge b a s e changes were r e q u i r e d t o upgrade the i n i t i a l system t o p r o v i d e such i n f o r m a t i o n . T h i s e x p e r i e n c e d e m o n s t r a t e s c l e a r l y t h a t one needs t o b u i l d i n t o t h e i n i t i a l k n o w l e d g e - b a s e d s y s t e m t h e f l e x i b i l i t y not o n l y t o m o d i f y the knowledge base (to be c o v e r e d more f u l l y i n a s e c t i o n on s y s t e m m a i n t e n a n c e ) but a l s o t o m o d i f y t h e b a s i c s t r u c t u r e and g o a l s of the system as r e q u i r e d i n t h e f u t u r e . How

Will

the

System Meet t h e s e

Expectations?

H a v i n g e s t a b l i s h e d the e x p e c t a t i o n s f o r t h e system, i m p l e m e n t a t i o n and maintenance become c r u c i a l .

the

issues

of

I m p l e m e n t a t i o n . Today, t h e r e a r e many o p t i o n s a v a i l a b l e t o t h e p r o s p e c t i v e a r c h i t e c t of a knowledge-based system. Developments i n



242


F i g u r e 3.

1)

NITRO

0.99

2)

NITRO-AROMATIC

0.99

3)

AROMATIC

0.99

4)

AROM-1-2-3-SUBST

0.76

5)

METHYL

0.72

6)

AMINE

0.60

7)

AMINE-TERTIARY

0.60

8)

AROM-1.2-SUBST

0.57

9)

AROM-1.3-SUBST

0.57

The r e s u l t s o f an e a r l y PAIRS i n t e r p r e t a t i o n compound "nitrotoluene".


f o r the


18. TOMELLINIETAL.


FUNCTIONALITY NITRO PASSED INITIAL EMPIRICAL FORMULA TEST * (Looking for asymmetric 0-N=0 stretch for a Nltro group in * the presence of an Alpha-withdrawing group: Note the positions * of the symmetric and asymmetric vibrations for these, vs those * for subseqquent queries. The withdrawing group causes an * increase in the coupling of the symmetric and asymmetric * vibrations, here, resulting in increased peak seperation * between the two:) PEAK QUERY ANY PEAK(S) POSITION: 1571 - 1620 INTENSITY: 7-10 WIDTH: SHARP TO BROAD ANSWER YES * (Looking for symmetric 0-N=0 stretch, in presence of Alpha* withdrawing group:) PEAK QUERY ANY PEAK(S) POSITION: 1276 - 1330 INTENSITY: 7 - 10 WIDTH: SHARP TO BROAD ANSWER--~~YES ACTION SET NITRO-ALPHA-WGROUP TO .500 CURRENT VALUE = .500 * (Begin saturated Nitro group tree. Looking for asymmetric 0-N=0 * stretch:) PEAK QUERY ANY PEAK(S) POSITION: 1541 - 1570 INTENSITY: 4-10 WIDTH: SHARP TO BROAD ANSWER NO * (Finished saturated nitro tree, begin unsaturated/aromatic* nitro tree-note lower the stretching frequencies, due to * conjugation: Looking for asymmetric 0-N=0 stretch:) PEAK QUERY ANY PEAK(S) POSITION: 1466 - 1549 INTENSITY: 7 - 10 WIDTH: SHARP TO BROAD ANSWER YES * (Looking for symmetric 0-N=0 stretch:) PEAK QUERY ANY PEAK(S) POSITION: 1291 - 1360 INTENSITY: 7 - 10 WIDTH: SHARP TO BROAD ANSWER YES ACTION---—SET NITRO-AROMATIC TO .250 CURRENT VALUE = .250 ACTION SET NITRO-UNSATURATED TO .250 CURRENT VALUE = .250 * (Narrowing peak-position interval searched, to converge on most * likely area for asymmetric stretch:) PEAK QUERY ANY PEAK(S) POSITION: 1500 - 1549 INTENSITY: 7-10 WIDTH: SHARP TO BROAD ANSWER YES ACTION-ADD .250 TO NITRO-UNSATURATED CURRENT VALUE = .500

(Trace Continues) F i g u r e 4.

A p a r t i a l t r a c e i n c l u d i n g d e s c r i p t i v e comments f o r the f u n c t i o n a l i t y " n i t r o " from t h e i n t e r p r e t a t i o n o f the s p e c t r a l d a t a o f n i t r o t o l u e n e .


243


244


s o f t w a r e d e s i g n t o o l s s i n c e t h e i n i t i a l development o f PAIRS have b r o u g h t about a h o s t o f new computer l a n g u a g e s , along with the i n t r o d u c t i o n o f " s h e l l " programs (10-12) . A b r i e f d i s c u s s i o n of t h e b a s i c components making up an e x p e r t s y s t e m i s i n o r d e r b e f o r e c o n s i d e r i n g the m e r i t s o f t r a d i t i o n a l languages v e r s u s s h e l l s . The h e a r t o f any e x p e r t s y s t e m i s t h e knowledge b a s e . The knowledge base i s a c o m p i l a t i o n o f f a c t s , o r d a t a , and r u l e s o f i n f e r e n c e , t o be a p p l i e d t o t h e p r o b l e m t o be s o l v e d . These r u l e s and f a c t s a r e p r o c e s s e d by an i n f e r e n c e e n g i n e . The i n f e r e n c e engine is a central unit which c o n t r o l s the selection and a p p l i c a t i o n o f " a p p r o p r i a t e " r u l e s and f a c t s , and i n some c a s e s , may r e v i s e t h e knowledge b a s e as new f a c t s are "deduced". An important c h a r a c t e r i s t i c of e x p e r t systems i s t h a t these two components a r e s e p a r a t e and d i s t i n c t e n t i t i e s . The i n t e r f a c e i s a software window t o t h e inference engine. A l l user/program i n t e r a c t i o n s , i n c l u d i n g d a t a i n p u t , i n f e r e n c e t r a c i n g , and the r e p o r t i n g of r e s u l t s , o c c u r t h r o u g h t h i s i n t e r f a c e . W h i l e not a f a c t o r i n program p e r f o r m a n c e , the q u a l i t y o f the i n t e r f a c e l a r g e l y d e t e r m i n e s how u s e f u l the system w i l l be i n p r a c t i c e . Which t o o l i s b e s t f o r d e v e l o p i n g e x p e r t systems? The answer i s t h a t i t d e p e n d s on t h e j o b t o be done and the resources available. How l a r g e i s the problem t o be solved? Are p r o f e s s i o n a l programmers a n d / o r knowledge e n g i n e e r s a v a i l a b l e t o c r e a t e t h e system? Is b r o a d hardware c o m p a t i b i l i t y an i s s u e ? Which i s more i m p o r t a n t , d e v e l o p m e n t t i m e o r s y s t e m p e r f o r m a n c e time? Is t h e s y s t e m t o be d i s t r i b u t e d ? How important i s cost? These q u e s t i o n s a r e some o f t h e c o n s i d e r a t i o n s i n t h e c h o i c e o f development and deployment t o o l s . An e x p e r i e n c e d programmer may e l e c t t o implement t h e s y s t e m " f r o m s c r a t c h " i n one o f many c u r r e n t l y a v a i l a b l e l a n g u a g e s . A t h o r o u g h d i s c u s s i o n o f the m e r i t s o f language c h o i c e s i s beyond the scope o f t h i s t e x t , however some g e n e r a l p o i n t s may be made. Twenty y e a r s ago, a r t i f i c i a l i n t e l l i g e n c e (AI) was c o n s i d e r e d a s u b j e c t f o r a c a d e m i c r e s e a r c h , not p r a c t i c a l a p p l i c a t i o n . A number o f l a n g u a g e s , most n o t a b l y LISP (12.) and P r o l o g (14.) , were c o n s i d e r e d the o n l y s u i t a b l e v e h i c l e s f o r A I . T r a d i t i o n a l l y , these have been offered as i n t e r p r e t e d languages, facilitating e x p l o r a t o r y AI r e s e a r c h and p r o v i d i n g t h e a b i l i t y f o r t h e program to modify i t s e l f while running. F o r programmers s k i l l e d i n t h e s e languages, they offered simpler expression of logic-driven programming t h a n e x i s t i n g p r o c e d u r a l l a n g u a g e s s u c h as FORTRAN, BASIC, and P a s c a l . P r o s p e c t i v e u s e r s s h o u l d be a d v i s e d however, t h a t b o t h P r o l o g and LISP, as w e l l as t h e o b j e c t - o r i e n t e d l a n g u a g e s d e s c r i b e d l a t e r , have a r e p u t a t i o n f o r c r e a t i n g CPU-intensive a p p l i c a t i o n s r e q u i r i n g l a r g e amounts of memory. As t h e t r a n s i t i o n f r o m l a b o r a t o r y t o market has o c c u r r e d , p r a c t i c a l c o n s i d e r a t i o n s have b r o u g h t a g r o w i n g t r e n d t o w a r d AI programming i n c o n v e n t i o n a l l a n g u a g e s (JL5.) , p a r t i c u l a r l y C and Pascal. These l a n g u a g e s o f f e r a much l a r g e r base o f programming t a l e n t t o d e s i g n and m a i n t a i n the programs, as w e l l as a b r o a d e r range o f hardware f o r i m p l e m e n t a t i o n . The r e s u l t i n g systems a r e g e n e r a l l y more compact and o f f e r much f a s t e r e x e c u t i o n speeds t h a n LISP and P r o l o g . S i z e and speed a r e p a r t i c u l a r l y i m p o r t a n t f o r microcomputer-based systems, f o r very l a r g e programs, or f o r systems r e q u i r i n g r a p i d r e s p o n s e . A d d i t i o n a l l y , the compilers f o r t h e s e l a n g u a g e s (as w e l l as t h e programming s k i l l s ) a r e n o r m a l l y a v a i l a b l e at reasonable c o s t . One a r e a r e c e i v i n g a g r e a t d e a l o f r e c e n t a t t e n t i o n i s t h e a p p l i c a t i o n o f o b j e c t - o r i e n t e d languages (16-18), as e x e m p l i f i e d by



18. TOMELLINI ETAL.


245

Smalltalk. These languages can offer greatly accelerated development due t o t h e i r h i g h l e v e l o f a b s t r a c t i o n and their i n h e r e n t l y r e u s a b l e code. They f e a t u r e a l a r g e body o f p r e d e f i n e d " o b j e c t c l a s s e s " and a s s o c i a t e d "methods", which may be expanded w i t h e a c h a p p l i c a t i o n . D e s i g n e d as an e x p l o r a t o r y programming environment, t h e s e a r e l a r g e l y i n t e r p r e t e d l a n g u a g e s . A l t h o u g h many p e o p l e a g o n i z e o v e r t h e c h o i c e o f a programming l a n g u a g e , a g e n e r a l r u l e i s t h a t u n l e s s some d e s i g n c r i t e r i o n d i c t a t e s a p a r t i c u l a r c h o i c e , t h e b e s t language t o use i s t h e one w i t h which t h e d e v e l o p e r i s most f a m i l i a r . V i r t u a l l y any mechanism can be implemented w i t h any l a n g u a g e . The d i f f i c u l t i e s a s s o c i a t e d w i t h becoming p r o f i c i e n t i n an u n f a m i l i a r language u s u a l l y outweigh any p o s s i b l e b e n e f i t s . Those i n d i v i d u a l s who l a c k e x t e n s i v e programming e x p e r i e n c e and/or t h e a i d o f a "knowledge e n g i n e e r " may e l e c t t o use an e x p e r t s y s t e m s h e l l t o implement t h e i r system. These programs p r o v i d e a ready-made framework f o r d e v e l o p i n g an e x p e r t system, hence t h e term " s h e l l " . A v a i l a b l e products d i f f e r g r e a t l y i n both f e a t u r e s and c o s t . The c o s t spans a range from l e s s t h a n $100 t o w e l l o v e r $10,000. S e r i o u s d e v e l o p e r s s h o u l d e x p e c t t o spend a t l e a s t $500 f o r base microcomputer systems, more f o r p r o d u c t s o f f e r i n g e x t e n d e d features and/or implemented on m i n i and superminicomputers. A d d i t i o n a l l y , d i s t r i b u t i o n o f t h e s y s t e m may require potential u s e r s t o p u r c h a s e t h e s h e l l i n t e r p r e t e r , o r may i n v o l v e l i c e n s i n g fees. W h i l e t h e s e programs d i f f e r g r e a t l y , t h e y s h a r e a common s e t of basic elements. A l l include one or more knowledge r e p r e s e n t a t i o n schemes which d e f i n e t h e s t r u c t u r e o f t h e knowledge base. A l l s u p p l y an i n f e r e n c e e n g i n e which can a p p l y one o r more methods o f i n f e r e n c e , s u c h as b a c k w a r d and f o r w a r d c h a i n i n g . F i n a l l y , a l l p r o v i d e some means o f t r a c i n g and o f t e n i n q u i r i n g i n t o the problem s o l v i n g p r o c e s s . S h e l l programs may a l l o w a competent s c i e n t i s t w i t h l i t t l e computer e x p e r i e n c e t o d e v e l o p a s y s t e m w i t h o u t t h e a i d o f computer professionals. In t h e hands o f a computer p r o f e s s i o n a l , t h e y a l l o w rapid development o f complex, user-friendly systems. They g e n e r a l l y u t i l i z e advanced ( f o r most chemist/programmers) database and inference t e c h n i q u e s , and provide sophisticated user interfaces. Indeed, i t i s d o u b t f u l t h a t most s c i e n t i s t s would be capable of d e v e l o p i n g a system u s i n g a c o n v e n t i o n a l language environment that i s comparable t o t h e more advanced shell offerings, w i t h o u t i n v e s t i n g a tremendous amount o f t i m e and effort. W h i l e t h e l i m i t e d c a p a b i l i t i e s o f e a r l i e r s h e l l programs made them u s e f u l l a r g e l y as a l e a r n i n g environment o r as a t o o l f o r system development by a domain e x p e r t , c u r r e n t shells offer sufficient features t o p r o v i d e a knowledge e n g i n e e r w i t h a framework f o r an advanced system. One f i n a l p o s s i b i l i t y i s t o use a combined a p p r o a c h , u s i n g an o b j e c t - o r i e n t e d language o r a s h e l l program f o r s y s t e m development or prototyping, and implementing the d e l i v e r y system in a c o n v e n t i o n a l l a n g u a g e , such as C o r P a s c a l . Particularly useful f o r l a r g e p r o j e c t s , t h i s a p p r o a c h l e t s e a c h t o o l do what i t does best. Maintenance. The l o n g - t e r m maintenance o f a knowledge-based system d e v e l o p e d t o a s s i s t r e s e a r c h e r s has b o t h s c i e n t i f i c and p r a c t i c a l implications. The s c i e n t i f i c i m p l i c a t i o n s stem from t h e f a c t t h a t t h e knowledge base i n most i f not a l l a r e a s o f s c i e n c e c o n t i n u e s t o grow w i t h t i m e . Even t h o s e a r e a s p e r c e i v e d t o be s t a g n a n t o f t e n



246


b e n e f i t f r o m a d v a n c e d t e c h n o l o g y and t h e r e f o r e change w i t h t i m e . J u s t as s c i e n t i s t s who do n o t keep up w i t h t h e d e v e l o p m e n t s i n their particular field are q u i c k l y o b s o l e t e , knowledge-based systems w h i c h a r e not u p d a t e d r e g u l a r l y i n f i e l d s where advances a r e t a k i n g p l a c e w i l l a l s o q u i c k l y become o b s o l e t e . I f the knowledge-based s y s t e m i s p r i m a r i l y used by r e s e a r c h e r s i n a g i v e n f i e l d , t h e c o n s e q u e n c e s o f u s i n g a n o n - m a i n t a i n e d s y s t e m can be especially serious. J u s t as a r e s e a r c h e r would q u e s t i o n an e x p e r t who has not kept up w i t h a f i e l d f o r many y e a r s , so t o o s h o u l d the r e s e a r c h e r be c a u t i o u s when u s i n g a knowledge-based s y s t e m t h a t has not been u p d a t e d t o i n c l u d e t h e l a t e s t a d v a n c e s , knowledge, and teachings i n a given area. The f a s t e r t h e a r e a o f e x p e r t i s e i s e x p a n d i n g , t h e g r e a t e r t h e chance t h a t t h e knowledge-based s y s t e m w i l l not be r e l i a b l e . The m a i n t e n a n c e o f a k n o w l e d g e - b a s e d s y s t e m i s a l s o o f practical importance. In t h i s c a s e , we a r e r e f e r r i n g n o t to u p d a t i n g t h e knowledge base but i n s t e a d t o u p d a t i n g the s o f t w a r e so t h a t i t can be run u s i n g hardware and s o f t w a r e a d v a n c e s as t h e y become a v a i l a b l e . I f the s o f t w a r e o f t h e knowledge-based system i s not s o f t w a r e and hardware c o m p a t i b l e w i t h the computers c u r r e n t l y used by researchers i n a given field, then the system is e s s e n t i a l l y not a v a i l a b l e t o the e n d - u s e r s and i n t e r e s t among t h e s e users w i l l e v e n t u a l l y vanish. Systems which a r e e a s i l y m a i n t a i n e d a r e t h e r e s u l t o f e f f o r t s made i n t h e d e s i g n s t a g e s o f d e v e l o p m e n t . Proper programming t e c h n i q u e s d i c t a t e t h e f o l l o w i n g (23.) : 1) . A l l s o u r c e

code s h o u l d be

2) . M e a n i n g f u l etc.

names s h o u l d

liberally be

chosen

3) . M o d u l a r ( s t r u c t u r e d ) programming be employed.

commented. for variables,

procedures,

u s i n g top-down d e s i g n

should

4) . Subprograms s h o u l d be made as g e n e r a l as p o s s i b l e t o a l l o w e a s y a d a p t a t i o n o f t h e code and s h o u l d " h i d e " t h e d e t a i l s o f i m p l e m e n t a t i o n from o t h e r program modules. 5) . Programs s h o u l d be w r i t t e n f o r maximum c l a r i t y , i . e . , t h e code s h o u l d be t h e most literal r e p r e s e n t a t i o n of the ideas possible. For ease of maintenance, t h i s p o i n t cannot be overemphasized. Additional points shown below:

specific

to

expert

system

design

include

those

6) . The use o f m e t a r u l e s (ZSL) , where a p p l i c a b l e , w i l l provide an o u t l i n e t o t h e knowledge base, a l o n g w i t h a m o d u l a r i t y t o t h e knowledge base s t r u c t u r e . 7) . The r u l e b a s e must be s t r u c t u r e d i n some way. This may i n v o l v e g r o u p i n g l i k e r u l e s t o g e t h e r and/or o r g a n i z i n g the r u l e base i n a h i e r a r c h i c a l f a s h i o n . A h i e r a r c h i c a l r u l e base provides some o f the advantages of metarules, namely, g e n e r a l i z e d upper l e v e l r u l e s may be e a s i l y " r e u s e d " . 8) . The

r u l e base i t s e l f

s h o u l d be commented where

necessary.


18.

TOMELLINIET AL.


247

9 ) . A s e p a r a t e i n f e r e n c e e n g i n e s h o u l d be used t o p r o c e s s t h e r u l e base, a l l o w i n g f o r m u l t i p l e i n f e r e n c e s t r a t e g i e s u s i n g t h e same rule base and p r o v i d i n g easy rule modification. A d d i t i o n a l l y , t h e i n f e r e n c e e n g i n e i t s e l f may be r e u s e d f o r other p r o j e c t s .


10).

Some a v a i l a b l e t o o l s p r o v i d e t r a n s l a t i o n several formats usable by d i f f e r e n t engines.

o f a r u l e base i n t o existing inference

Q u e s t i o n s c o n c e r n i n g how b e s t t o m a i n t a i n a p r o p o s e d ( o r e x i s t i n g ) k n o w l e d g e - b a s e d s y s t e m , who w i l l be r e s p o n s i b l e f o r m a i n t a i n i n g t h e s y s t e m , and how l o n g - t e r m m a i n t e n a n c e w i l l be f i n a n c e d s h o u l d a l l be r e c t i f i e d p r i o r t o b e g i n n i n g development on such a system. I t i s o n l y by d e s i g n i n g f o r p r o p e r m a i n t e n a n c e t h a t a knowledge-based system c a n age g r a c e f u l l y , c o n t i n u e t o p e r f o r m a t reasonable levels o f e x p e r t i s e , and c o n t i n u e t o p r o t e c t t h e o r i g i n a l i n v e s t m e n t i n i t s development. Publishability i n Scientific Journals. After determining the e x p e c t a t i o n s and t h e i m p l e m e n t a t i o n p r o c e d u r e s , a t l e a s t one o t h e r v i t a l i s s u e needs t o be a d d r e s s e d . The a b i l i t y t o p u b l i s h t h e r e s u l t s o f ones e f f o r t s i s o f g r e a t i m p o r t a n c e to analytical chemists. The p u b l i s h a b i l i t y o f t h e d e v e l o p m e n t o f k n o w l e d g e - b a s e d systems r a i s e s a number o f i n t e r e s t i n g i s s u e s . 1) .

How s h o u l d one judge whether o r n o t t o p u b l i s h t h e r e s u l t s o f development o f a knowledge-based system?

2) .

Should p u b l i c a t i o n s be aimed a t c h e m i s t s who a r e l i k e l y t h e system, computer s p e c i a l i s t s , o r both?

3) .

How s h o u l d improvements t o e x i s t i n g knowledge-based systems be t r e a t e d , a s p u b l i s h a b l e r e s e a r c h d e v e l o p m e n t s o r as n o n p u b l i s h a b l e program m o d i f i c a t i o n s ?

4) .

S h o u l d t h e p u b l i s h a b i l i t y o f e f f o r t s i n t h e a r e a o f knowledgeb a s e d s y s t e m s be t r e a t e d u s i n g t h e same c r i t e r i a as o t h e r areas of chemical research?

5) .

What i s l i k e l y t o happen i f e f f o r t s a r e made t o l i m i t t h e p u b l i s h a b i l i t y of t h e r e s u l t s o f e f f o r t s i n t h i s area?

t o use

A c r i t i c a l concern i s t h a t i t takes a g r e a t d e a l o f time t o d e v e l o p a s u b s t a n t i a l and u s e f u l k n o w l e d g e - b a s e d s y s t e m . I t can e a s i l y t a k e y e a r s from b e g i n n i n g development o f such a s y s t e m u n t i l t h e f i r s t p u b l i c a t i o n o f t h e r e s u l t s o f one's e f f o r t s . Clearly t h i s r a t e o f . p u b l i s h i n g w i l l n o t be s u f f i c i e n t t o m a i n t a i n a successful academic o r i n d u s t r i a l research career. Further c o m p l i c a t i n g t h e i s s u e , a s i n g l e p a p e r on any g i v e n s u b j e c t m a t t e r t e n d s t o go v i r t u a l l y u n n o t i c e d i n t h e w e a l t h o f i n f o r m a t i o n t h a t is published i n s c i e n t i f i c journals. Thus t h e r e s e a r c h e r i s f a c e d w i t h a r a t h e r s e r i o u s problem, e s p e c i a l l y i f e d i t o r s and r e v i e w e r s b e l i e v e t h a t s y s t e m improvements a r e n o t p u b l i s h a b l e r e s e a r c h . E d i t o r s , r e v i e w e r s , and t h o s e i n v o l v e d i n c r e a t i n g t h e s e systems w i l l need t o a d d r e s s t h e q u e s t i o n s r a i s e d above i f p r o g r e s s i n t h i s area i s t o continue.

American Chemical Society Library 1155 15th St., N.W. Washington, O.C. 20036 Hohne and Pierce; Expert System Applications in Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 1989.

248



Conclusions Improving c o m p u t e r - a s s i s t e d s p e c t r o s c o p i c i n t e r p r e t a t i o n s c o n t i n u e s t o be one o f t h e g o a l s o f o u r r e s e a r c h a c t i v i t i e s . As t h e e f f o r t s have evolved, many issues have h a d t o be addressed and reconsidered. I t i s c r i t i c a l t o c o n s i d e r a t t h e o u t s e t what t h e s y s t e m i s t o do. What a r e t h e e x p e c t a t i o n s ? F o r whom i s t h e system designed? How l a r g e ( u s e f u l ) w i l l i t be? ( i . e . , i s i t a " b a s i c r e s e a r c h " e f f o r t o r i s i t an e f f o r t t o d e v e l o p a u s a b l e system?). Next, c o n s i d e r a t i o n must be g i v e n t o how t h e s y s t e m w i l l meet t h e s e e x p e c t a t i o n s . Fundamental t o t h e s e i s s u e s i s whether o r not t h e knowledge base w i l l be made a v a i l a b l e and m o d i f i a b l e . How one will maintain the system and r e p o r t a l t e r a t i o n s and developments a r e other c o n s i d e r a t i o n s . By a d d r e s s i n g a l l o f t h e s e i s s u e s a t t h e a p p r o p r i a t e t i m e s , d e v e l o p e r s and u s e r s o f knowledgeb a s e d systems w i l l be a b l e t o maximize t h e i r b e n e f i t s .

Literature Cited 1. 2. 3.

4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20.

Watson, H.J; Mann, R.I. Journal of Information Systems Management 1988, 5(4), 39. Woodruff, H.B.; Munk, M.E. J . Org. Chem. 1977, 42, 1761. Wipke, W.T.; Braun, H-.; Smith, G.; Choplin, F . ; Sieber, W. In Computer-Assisted Organic Synthesis; Wipke, W.T.; Howe, W. J , Eds.; American Chemical Society: Washington, D.C., 1977; pp 97-127. Woodruff, H.B.; Smith, G.M. Anal. Chem 1980, 52, 2321. Tomellini, S.A.; Saperstein, D.D.; Stevenson, J . M . ; Smith, G.M.; Woodruff, H.B.; Seelig, P.F. Anal. Chem. 1981, 53, 2367. Tomellini, S.A.; Stevenson, J . M . ; Woodruff, H.B.; Hartwick, R.A. Anal. Chim. Acta 1984, 162, 227. Tomellini, S.A.; Hartwick, R.A.; Woodruff, H.B. A p p l . Spectrosc. 1985, 39, 331. Saperstein, D.D. Appl. Spectrosc. 1986, 40, 344. Wythoff, B . J . ; Buck, C . F . ; Tomellini, S.A. Anal. Chim. Acta, 1989, 217, 203. Citrenbaum, R.; Geissman, J.R.; Schultz, R. AI Expert 1987, 2(9), 30. Arcidiacono, T. PC Tech Journal 1988, 6(5), 44. Fontana, M.; Zeimetz, J . PC Tech Journal 1988, 6(5), 63. Coffee, P. C. AI Expert 1988, 3(3), 38. Clocksin, W. BYTE 1987, 12(9), 147. Coffee, P.C.; Strauss, D.J. AI Expert 1988, 3(5), 38. Stein, J . Dr. Dobb's Journal of Software Tools for the Professional Programmer 1988, 13(3), 18. Thompson, B.; Thompson, B. AI Expert 1987, 2(9), 15. Ramamoorthy, C.V.; Sheu, P.C. IEEE Expert 1988, 3(3), 9. Martin, J.; McClure, C. Structured Techniques for Computing, Prentice-Hall Inc., Englewood C l i f f s , N . J . , 1985. Nilsson, N.J. Principles of A r t i f i c i a l Intelligence, Tioga Publishing Co., Palo Alto, Ca., 1980.

RECEIVED July 21, 1989


Developing Knowledge-Based Systems - ACS Symposium Series

Recommend Documents