Verification and Validation of Decision Support Expert Systems - ACS

Chapter 10

Verification and Validation of Decision Support Expert Systems Chemical Process Risk Management in International Operations

Downloaded by RUTGERS UNIV on March 3, 2016 | http://pubs.acs.org Publication Date: September 1, 1989 | doi: 10.1021/bk-1989-0408.ch010

Myron J. Miller Factory Mutual Engineering, Norwood MA 02062

Expert Systems are being designed or considered for both on-line process control and off-line decision support i n the handling of hazardous materials. There are possibilities for both positive and negative impacts on chemical industry fire/explosion loss potentials. We will review some guidelines and tools available for the verification and validation (V&V) of expert systems. Finally, we will illustrate some of the points discussed by describing the development of a prototype expert system for hazardous materials classification and loss prevention engineering decision support. OBJECTIVE - EARLY RECOGNITION OF V & V NEEDS We hope to stimulate an interest i n a very interesting and challenging subject with much work yet to be done. Consideration of V & V questions at an early stage of expert systems development should improve the chances of producing systems that are useful, safe and acceptable i n the marketplace. BACKGROUND Catastrophic accidents have been a lesson i n geography. Flixborough, Seveso, Feysin, Mexico City, Bhopal, Wesseling, Chernobyl, Schweizerhalle, Pampa, Norco and Piper Alpha are but a few of the recent incidents. Each issue of the Financial Times' World Insurance Report adds a few more to the l i s t . Together they have had a major impact on both government and industry. At the same time the litany of less visible incidents continues. For instance, the Factory Mutual (FM) System Loss Prevention Publication "Programmable Logic Controllers" (1) indicates that i n the period 1980-85 there were 729 losses involving programmable controllers with a net FM l i a b i l i t y of about $49 M i l l i o n . The potential exists for both positive and negative impacts of expert systems technology on industry loss experience. 0097-6156/89/0408-0126$06.25/0 © 1989 American Chemical Society

In Expert System Applications in Chemistry; Hohne, Bruce A., et al.; ACS Symposium Series; American Chemical Society: Washington, DC, 1989.

10.

MILLER

VeriJkatwnandValulatwnofDeciswnS

127

S i n c e many f a i l u r e s o f p r o c e s s c o n t r o l and p r o t e c t i v e systems a r e due t o human element f a i l u r e s , r e l i a b l e a u t o m a t i o n and d e c i s i o n s u p p o r t systems c a n h e l p t o c o n t r o l l o s s e s . On t h e o t h e r hand, i n a d e q u a t e d e s i g n , deployment o r c o n f i g u r a t i o n c o n t r o l can cause o r c o n t r i b u t e t o s i g n i f i c a n t l o s s e s . A r t i f i c i a l i n t e l l i g e n c e concepts and e x p e r t systems t e c h n o l o g y h o l d f o r t h t h e promise o f s i g n i f i c a n t improvements i n t h e s a f e t y o f c h e m i c a l p r o c e s s e s , b u t o n l y i f t h e s e t o o l s themselves a r e s a f e and r e l i a b l e . V e r i f i c a t i o n and V a l i d a t i o n (V&V), T e s t i n g and E v a l u a t i o n (T&E) w i l l be n e c e s s a r y throughout t h e e n t i r e l i f e c y c l e o f t h e system i f maximum b e n e f i t s a r e t o be obtained.


DEFINITIONS - VERIFICATION, VALIDATION, LIFECYCLE V e r i f i c a t i o n i s t h e p r o c e s s o f t e s t i n g and e v a l u a t i o n o f b o t h t h e s o f t w a r e and t h e knowledge base t o d e t e r m i n e i f they work p r o p e r l y t o g e t h e r and a c c o r d i n g t o s p e c i f i c a t i o n s . V a l i d a t i o n i s t h e p r o c e s s o f d e t e r m i n i n g whether t h e system a c t u a l l y h e l p s t h e u s e r a s i t was o r i g i n a l l y i n t e n d e d - and w i l l c o n t i n u e t o h e l p him i n t h e f u t u r e . V a l i d a t i o n s h o u l d a l s o make s u r e t h a t t h e l i m i t s o f a p p l i c a b i l i t y a r e d e f i n e d and t e s t e d , t h e l e v e l o f c o n f i d e n c e s p e c i f i e d and t h e consequences o f f a i l u r e due t o m i s u s e emphasized. D o l o r e s W a l l a c e o f t h e N a t i o n a l Computer and T e l e c o m m u n i c a t i o n s L a b o r a t o r y s u g g e s t s t h a t s o f t w a r e V&V be t r e a t e d a s a s i n g l e e n t i t y encompassing v e r i f i c a t i o n , v a l i d a t i o n and t e s t i n g . Formal d e f i n i t i o n s a r e found i n FIPSPUB 101. The L i f e c y c l e o f a system s h o u l d i n c l u d e t h e whole spectrum o f a c t i v i t i e s from t h e c r i t i c a l " d e s i g n f o r t e s t i n g " phase t o t h e f i n a l d e c i s i o n on when t h e system s h o u l d be abandoned. CURRENT A I ACTIVITIES IN CHEMICAL ENGINEERING Much i s happening a l r e a d y . More i n t e r a c t i o n between those s t u d y i n g c h e m i c a l s and t h e i r r e a c t i o n s , and t h o s e who u l t i m a t e l y d e s i g n and r u n - (and i n s u r e ) - t h e p r o c e s s p l a n t s would be b e n e f i c i a l . A l i s t i n g o f t i t l e s from some o f t h e r e c e n t A r t i f i c i a l I n t e l l i g e n c e / E x p e r t Systems s e s s i o n s o f AIChE m e e t i n g s i s appended t o g i v e you some i d e a o f t h e i n t e r e s t s o f your c o u n t e r p a r t s i n t h e C h e m i c a l E n g i n e e r i n g Community. Most o f t h e s e p a p e r s a r e a v a i l a b l e from t h e E n g i n e e r i n g S o c i e t i e s L i b r a r y i n New Y o r k C i t y , b u t have n o t been p u b l i s h e d i n book form by AIChE a s t h e ACS has done. Other c h e m i c a l e n g i n e e r i n g a p p l i c a t i o n s o r i e n t e d a c t i v i t i e s a r e a l s o i n e v i d e n c e . Ohio S t a t e U n i v e r s i t y conducted a summer "Workshop on A p p l i c a t i o n s of A r t i f i c i a l I n t e l l i g e n c e Tools i n the Process C o n t r o l I n d u s t r y " . A n o t h e r s e s s i o n , on " E x p e r t Systems i n P r o c e s s E n g i n e e r i n g & P r o c e s s Development, D e s i g n , C o n t r o l and O p e r a t i o n s " , was o r g a n i z e d by P r o f e s s o r George Stephanopoulos a t MIT. A s e s s i o n on " A r t i f i c i a l I n t e l l i g e n c e i n P r o c e s s E n g i n e e r i n g " was a l s o


128

EXPERT SYSTEM APPLICATIONS IN CHEMISTRY


s c h e d u l e d f o r t h e August 88 meeting A r t i f i c i a l I n t e l l i g e n c e (AAAI).

o f t h e American A s s o c i a t i o n f o r

There i s much a c t i v i t y , b o t h p u b l i s h e d and u n p u b l i s h e d , on e x p e r t systems a p p l i c a t i o n s i n c h e m i c a l p r o c e s s a n a l y s i s , d e s i g n and c o n t r o l . A paper i n Computers & C h e m i c a l E n g i n e e r i n g t i t l e d "An E x p e r t System Approach To M a l f u n c t i o n D i a g n o s i s i n C h e m i c a l P l a n t s " (2a) i s one o f t h e Ohio S t a t e e f f o r t s t h a t has been p u b l i s h e d . R e l a t e d a r t i c l e s a r e a l s o a p p e a r i n g i n t h e J o u r n a l s Computers i n C h e m i s t r y and C o m p u t a t i o n a l C h e m i s t r y . The J o u r n a l o f C o m p u t a t i o n a l C h e m i s t r y r e c e n t l y c o n t a i n e d an a r t i c l e by Klopman and Raychaudhury a t CASE Western Reserve d e s c r i b i n g a program c a l l e d Computer-Automated S t r u c t u r e E v a l u a t i o n (CASE)" (2b). Wu and L i d s k y a t MIT have p u b l i s h e d a r e p o r t (3) on t h e i r P r o l o g based system d e s i g n e d t o a i d n u c l e a r power p l a n t o p e r a t o r s t o f u n c t i o n w i t h i n NRC g u i d e l i n e s . Real-Time p r o c e s s c o n t r o l and d i a g n o s t i c e x p e r t systems a r e o n l y one a r e a where improved l o s s c o n t r o l c a n r e s u l t . The d e c i s i o n s r e l a t i n g t o t h e d e s i g n and p r o t e c t i o n o f s a f e f a c i l i t i e s a r e a l s o a r e a s e q u a l l y amenable t o t h e a p p l i c a t i o n o f Improved r i s k management d e c i s i o n a i d s . A paper by V o l k e r P i l z ( 4 ) o u t l i n e s many o f t h e c h e m i c a l r e a c t i v i t y , c o m p a t a b i l i t y and r e g u l a t o r y c o n s i d e r a t i o n s t h a t must be addressed i n t h e d e s i g n and p r o t e c t i o n of f a c i l i t i e s f o r s t o r i n g and h a n d l i n g hazardous c h e m i c a l s . Expert Systems would be u s e f u l f o r database a c c e s s , h a z a r d e s t i m a t i o n and d e c i s i o n making f o r b o t h n o r m a l and emergency o p e r a t i o n s . The f o c u s o f t h i s paper i s on a p p l i c a t i o n s o f a r t i f i c i a l i n t e l l i g e n c e t o improve i n d u s t r y l o s s e x p e r i e n c e . C o n s i d e r i n g s a f e t y and system v e r i f i c a t i o n , v a l i d a t i o n and t e s t i n g r e q u i r e m e n t s i n t h e e a r l i e s t s t a g e s o f m a t e r i a l s e v a l u a t i o n and p r o c e s s d e s i g n w i l l help a t t a i n t h i s goal. PURPOSE - AN OVERVIEW OF V & V The purpose here i s n o t t o p r e s e n t a t u t o r i a l on s o f t w a r e r e l i a b i l i t y o r q u a l i t y a s s u r a n c e , b u t t o pose some o f t h e q u e s t i o n s r e l a t i n g t o t h e v e r i f i c a t i o n and v a l i d a t i o n o f knowledge-based systems, t o o u t l i n e some o f t h e c u r r e n t r e s e a r c h i n t h i s a r e a , and review a few p i e c e s o f recent l i t e r a t u r e . I n t h e p a s t t h r e e y e a r s A I Technology has spawned a whole new p u b l i s h i n g i n d u s t r y . I t i s now a l m o s t i m p o s s i b l e t o r e a d e v e r y t h i n g b e i n g p u b l i s h e d about A I and E x p e r t Systems. A I Magazine, IEEE E x p e r t , E x p e r t Systems, A I J o u r n a l , A l / E x p e r t , I n t e r n a t i o n a l J o u r n a l o f I n t e l l i g e n t Systems, I n t e r n a t i o n a l J o u r n a l of L o g i c Programming and t h e I n t e r n a t i o n a l J o u r n a l o f E x p e r t Systems R e s e a r c h and A p p l i c a t i o n s a r e o n l y a s a m p l i n g o f j o u r n a l s i n E n g l i s h . Conference p r o c e e d i n g s , b o t h U.S. and f o r e i g n , a r e p r o l i f e r a t i n g r a p i d l y ( 5 , 6 , 7 ) . A t t h e same t i m e , much r e l e v a n t m a t e r i a l i s b e i n g p u b l i s h e d under t h e h e a d i n g s o f c o g n i t i v e s c i e n c e , c o m p u t a t i o n a l l i n g u i s t i c s , and a wide v a r i e t y o f applications-oriented t i t l e s .


10. MILLER

Verification and Validation of Decision Support ExpeH Systems 129

U n i v e r s i t i e s w o r l d - w i d e a r e p u b l i s h i n g a s t e a d y s t r e a m o f papers and d i s s e r t a t i o n s . One o f p a r t i c u l a r n o t e ( 8 ) , d e s c r i b i n g an e x p e r t system c a l l e d "LEILA" f o r " E s t i m a t i n g The R a t e s o f C h e m i c a l R e a c t i o n s " , was s p o t t e d i n t h e l i s t i n g s p u b l i s h e d by t h e ACM SIGART. There a r e a l s o r e a l l i v e e x p e r t systems b e i n g s o l d c o m m e r c i a l l y . An example i s one c a l l e d "SOPHIE" ( 9 ) , w h i c h p r o v i d e s a s y s t e m a t i c method f o r c h o o s i n g p r o c e d u r e s f o r a n a l y z i n g risk. I t i s based upon t h e p u b l i c a t i o n B a t t e l l e produced f o r t h e AIChE s Center f o r Chemical Process S a f e t y (10). 1


INTERNATIONAL FACTORS The use o f e x p e r t systems f o r b o t h o n - l i n e and o f f - l i n e r i s k a n a l y s i s and d e c i s i o n s u p p o r t becomes much more c o m p l i c a t e d i n an i n t e r n a t i o n a l environment. Human i n p u t and o u t p u t i n t e r p r e t a t i o n becomes a much more c h a l l e n g i n g e x e r c i s e . The u s e r model upon w h i c h t h e system depends most c e r t a i n l y w i l l have t o a c c o u n t f o r b o t h l i n g u i s t i c and c u l t u r a l f a c t o r s i f t h e system i s t o o p e r a t e r e l i a b l y and e f f e c t i v e l y . M u l t i c u l t u r a l human f a c t o r s must c e r t a i n l y be a d d r e s s e d i f we a r e t o succeed i n i n t e r n a t i o n a l t r a d e and m u l t i n a t i o n a l e n t e r p r i s e . About a q u a r t e r o f the w o r l d ' s p o p u l a t i o n speaks some E n g l i s h j u s t d i f f e r e n t v e r s i o n s . Q u e s t i o n s o f t e r m i n o l o g y and i n t e r p r e t a t i o n abound. T h r e e - f o u r t h s o f t h e w o r l d ' s p o p u l a t i o n speaks no E n g l i s h a t a l l , and t h e y c o n s t i t u t e t h e l a r g e s t p o t e n t i a l market f o r f u t u r e development. Of c o u r s e , p h y s i c a l c h e m i s t s and c h e m i c a l p h y s i c i s t s o f t e n have t r o u b l e communicating due t o t h e t e c h n i c a l j a r g o n and d i f f e r i n g p e r s p e c t i v e s . The same i s t r u e o f other d i s c i p l i n e s . Machine t r a n s l a t i o n o f " n a t u r a l " l a n g u a g e s i s i n my e s t i m a t i o n t h e r e a l t e s t of A I . Research p u b l i c a t i o n s are i n c r e a s i n g i n both q u a n t i t y and q u a l i t y and t h e C e n t e r f o r Machine T r a n s l a t i o n a t C a r n e g i e M e l l o n U n i v e r s i t y i s g i v i n g c u r r e n t e f f o r t s some f o c u s . Recent developments such as t h e Monroe keyboard f o r e n t e r i n g C h i n e s e c h a r a c t e r s ( B o s t o n Globe 7/25/88) i s o n l y one example o f s i g n i f i c a n t r e c e n t p r o g r e s s toward m u l t i l i n g u a l computing and machine t r a n s l a t i o n . SOFTWARE SYSTEMS SAFETY - TRIPLE REDUNDANCY FOR

SOFTWARE?

M i l STD-882B (11) o u t l i n e s t h e g e n e r a l c o n c e p t s o f Systems S a f e t y . N o t i c e 1 (12) appended t o t h a t document s p e c i f i c a l l y c o v e r s r e q u i r e m e n t s f o r s o f t w a r e systems s a f e t y . The A i r F o r c e a l s o has a S o f t w a r e Systems S a f e t y Handbook ( 1 3 ) . A number o f r e c e n t a r t i c l e s i n Hazard P r e v e n t i o n ( 1 4 ) , t h e J o u r n a l o f t h e Systems S a f e t y S o c i e t y , expanded on t h e t e c h n i c a l c o n t e n t o f t h i s handbook. The c o n c e p t s were c o v e r e d i n a more e x p l i c i t and u s e f u l manner by E r w i n S c h o i t s c h o f the A u s t r i a n R e s e a r c h C e n t e r S e i b e r s d o r f i n a r e c e n t f o r t y - t w o page, t w o - p a r t , a r t i c l e ( 1 5 ) .


130


H e i n z Luck and Udo S c h l o s s a r e k (16) a t D u i s b e r g U n i v e r s i t y d i s c u s s e d the r e l a t i v e m e r i t s o f a " p e r f e c t i o n i s t " v e r s u s a " f a u l t t o l e r a n t " approach i n the d e s i g n o f " S o f t w a r e C o n t r o l l e d F i r e D e t e c t i o n Systems". T h i s document a l s o d i s c u s s e s the c o n c e p t o f t r i p l e redundancy o f s o f t w a r e , e m p l o y i n g d i f f e r e n t a l g o r i t h m s , d i f f e r e n t computer l a n g u a g e s , d i f f e r e n t a n a l y s t s and d i f f e r e n t programmers f o r p a r a l l e l c h a n n e l s w i t h the same i n p u t / o u t p u t o b j e c t i v e s . T h i s i s a k i n t o u s i n g the d e v i c e s o f d i f f e r e n t m a n u f a c t u r e r s , employing d i f f e r e n t p r i n c i p l e s i n a redundant hardware system t o d e c r e a s e the chances o f common mode f a i l u r e .


A document c a l l e d "The E n g i n e e r s R e s p o n s i b i l i t y f o r Computer Based D e c i s i o n s " ( 1 7 a ) , p u b l i s h e d by the I n s t i t u t i o n o f C h e m i c a l E n g i n e e r s , recommends t r i p l e redundancy - w i t h v o t i n g - f o r software i n " c r i t i c a l " a p p l i c a t i o n s . f

S e v e r a l o f FM s L o s s P r e v e n t i o n Data P u b l i c a t i o n s ( 1 , 17B, 17C) d i s c u s s the c o n c e p t o f t r i p l y - r e d u n d a n t , f a u l t - t o l e r a n t , h i g h r e l i a b i l i t y h a r d w a r e / s o f t w a r e systems f o r m a n u f a c t u r i n g o p e r a t i o n s . R i s k a n a l y s i s and systems r e l i a b i l i t y r e s e a r c h i s c u r r e n t l y underway t o d e v e l o p b e t t e r g u i d e l i n e s f o r the d e s i g n and a p p l i c a t i o n o f r e l i a b l e p r o c e s s c o n t r o l systems. None o f the documents n o t e d above c o v e r s the q u e s t i o n s s p e c i f i c t o V&V f o r knowledge-based e x p e r t systems, a l t h o u g h many o f the c o n c e p t s d i s c u s s e d a r e a p p l i c a b l e . And t r i p l e redundancy i s u s e l e s s u n l e s s the top l e v e l problems o f s a f e t y have been a d d r e s s e d i n system s p e c i f i c a t i o n ' s . THE

FUTURE OF SOFTWARE SYSTEMS SAFETY

R&M 2000, the A i r F o r c e R e l i a b i l i t y and M a i n t a i n a b i l i t y A c t i o n P l a n does n o t d i s c u s s the u n i q u e problems (and p o t e n t i a l ) o f e x p e r t s y s t e m s , p a r t i c u l a r l y those w h i c h r e l y on the m a s s i v e "knowledge b a s e s " w h i c h can now be d e l i v e r e d on o p t i c a l d i s c . Nor have t h e y a d d r e s s e d the problems (and p o t e n t i a l ) a s s o c i a t e d w i t h p a r a l l e l p r o c e s s i n g , n e u r a l networks and "systems t h a t l e a r n " . D i s c u s s i o n s w i t h the Navy's r e p r e s e n t a t i v e t o the T r i s e r v i c e S o f t w a r e Systems S a f e t y Group and the NBS (Now NIST) r e p r e s e n t a t i v e t o the Compass 88 Conference I n d i c a t e t h a t the impact ( b o t h p o s i t i v e and n e g a t i v e ) o f A I on s o f t w a r e s a f e t y i s r e c o g n i z e d b u t has not as y e t been documented o r i n c o r p o r a t e d i n t o an a c t i o n p l a n . The s t a t e o f the a r t can b e s t be summarized by a r e c e n t RFP from Rome A i r F o r c e Base (19) c a l l i n g f o r "The development o f a s e l f - d i a g n o s i n g , s e l f - r e p a i r i n g , a c t i v e , i n t e l l i g e n t d a t a base c a p a b i l i t y whose a r c h i t e c t u r e w i l l u t i l i z e knowledge e n g i n e e r i n g t e c h n i q u e s , system c o n s t r a i n t s and i n t e l l i g e n t i n t e g r i t y r u l e s i n o r d e r t o p r o v i d e a d a t a management c a p a b i l i t y r e s i l i e n t t o l o g i c a l f a u l t s , e r r o n e o u s and s p u r i o u s I n p u t s / u p d a t e s and I n t e g r i t y problems caused by m a l i c i o u s updates o r i n c o m p l e t e d a t a . " Q u i t e a wish l i s t .


10.

MILLER

Verifeation and Validation ofDecision Support Expert Sy

An o v e r v i e w o f p r o c e d u r a l s o f t w a r e v e r i f i c a t i o n i s c o v e r e d by M i l l ( 2 0 a ) . Q u i r k (20b) c o v e r s V&V o f r e a l time s o f t w a r e and D e M i l l e , e t a l (20c) c o v e r s t h e g e n e r a l s u b j e c t o f s o f t w a r e t e s t i n g and evaluation.


There i s a l s o an i n c r e a s i n g body o f l i t e r a t u r e ( 2 1 , 22, 23) c o n c e r n i n g t h e r e l i a b i l i t y o f programmable c o n t r o l l e r s ( o r programmable l o g i c c o n t r o l l e r s (PLC's) a s they a r e now c a l l e d ) . I n t h e UK t h e s e f a l l under t h e H e a l t h and S a f e t y E x e c u t i v e ' s (HSE) g u i d e l i n e s f o r programmable e l e c t r o n i c systems ( P E S ) . See C h e m i c a l E n g i n e e r (UK), f o r a d i s c u s s i o n ( 2 4 ) . The book (25) "Programmable C o n t r o l l e r s : Theory and I m p l e m e n t a t i o n " i s a good o v e r v i e w w h i c h i n c l u d e s a c h a p t e r on " A r t i f i c i a l I n t e l l i g e n c e and PLC Systems". More and more d e v i c e s , such a s Foxboro's EXACT system, i n c l u d e " a r t i f i c i a l i n t e l l i g e n c e based" a d a p t i v e , s e l f - t u n i n g f e a t u r e s . V&V BY AND FOR EXPERT SYSTEMS P e t e r P o l i t a k i s * PhD t h e s i s e n t i t l e d " E m p i r i c a l A n a l y s i s f o r E x p e r t Systems" (26) i s an e x c e l l e n t i n t r o d u c t i o n t o t h e s u b j e c t o f e x p e r t systems V&V. P o l i t a k i s and Weiss (27) c o v e r t h e use o f case e x p e r i e n c e t o r e f i n e t h e r u l e s o f a model i n a m e d i c a l d i a g n o s t i c s system. An e x c e l l e n t s u r v e y a r t i c l e appeared i n A l / E x p e r t ( 2 8 ) . Papers by S t a c h o w i t z , e t a l (29a, b, c , d) p r o v i d e a a good o v e r v i e w o f most of t h e V&V component q u e s t i o n s . The e f f o r t uses P r o l o g f o r i t s i n h e r e n t l o g i c c h e c k i n g p o t e n t i a l . The elements o f Lockheed's V & V e x p e r t system EVA ( E x p e r t Systems V a l i d a t i o n A s s o c i a t e ) a r e shown i n F i g u r e 1. Three o t h e r papers from t h e SOAR Conference a l s o c o v e r V&V c o n c e r n s ( 3 0 , 3 1 , 3 2 ) . The book " A l g o r i t h m i c Program Debugging" by Ehud S h a p i r o (33) i s i n t e r e s t i n g f o r i t s i n - d e p t h d e s c r i p t i o n o f debugging algorithmms written i n Prolog. A 1986 IC0T r e p o r t (35) from Japan d e s c r i b e s "ARGUS/V: f o r V e r i f i c a t i o n o f P r o l o g Programs".

A System

F o r t h o s e o f you i n t e r e s t e d i n P r o l o g , t h e book " P r o l o g Programming i n Depth" ( 3 4 ) , has a good d i s c u s s i o n o f program debugging and v e r i f i c a t i o n , a s w e l l as an example r e p r e s e n t a t i o n i n P r o l o g o f a connectivity table f o r molecular structure representation. A paper by R i c h a r d s o n and Wong (36) i n d i c a t e s t h a t t h e KBS l i f e c y l e V&V p r o c e s s i s now b e i n g f o r m a l i z e d and w i l l be t h e f i r s t t o i n c o r p o r a t e new q u a l i t y s t a n d a r d s f o r s o f t w a r e . They s t a t e t h a t the purpose o f V&V i s "To ensure t h a t s o f t w a r e i s 1) d e s i g n e d t o be t e s t a b l e , 2) completed a c c o r d i n g t o d e f i n e d s p e c i f i c a t i o n s w h i c h l e a d t o t e s t a b i l i t y , and 3) t e s t e d a c c o r d i n g t o r e q u i r e m e n t s . "


131

In Expert System Applications in Chemistry; Hohne, Bruce A., et al.; ACS Symposium Series; American Chemical Society: Washington, DC, 1989. Figure 1. Validation functionality. (Adapted from ref. 29b.)


10.

MILLER

VeriJkationandValidationofDecision Support Expert Systems 133

A p p l i c a t i o n o f V&V t o e x p e r t systems f o r n u c l e a r power p l a n t s i s d e s c r i b e d i n an E l e c t r i c Power R e s e a r c h I n s t i t u t e r e p o r t ( 3 7 ) . T a b l e 5.9-1 i n t h a t r e p o r t i n d i c a t e s t h e "V&V L i f e c y c l e Phases" f o r a number o f e x p e r t system t y p e s . A g a i n , t h e f o c u s i s on L i f e c y c l e V&V.


We w i l l n o t d i s c u s s t h e problems o f bugs, v i r u s e s , T r o j a n h o r s e s , l o g i c time bombs and h a c k e r s . You s h o u l d a l l r e c o g n i z e by now t h e q u e s t i o n s o f s o f t w a r e s e c u r i t y t h a t must be a d d r e s s e d i n b o t h t h e d e s i g n and c o n f i g u r a t i o n c o n t r o l p r o c e s s e s . F a c t o r s t h a t might a f f e c t t h e performance o f software/hardware/humanware systems a r e b e i n g a d d r e s s e d i n d e p t h a t a n n u a l c o n f e r e n c e s o f t h e Computer S e c u r i t y I n s t i t u t e , a s w e l l a s by v a r i o u s government and i n d u s t r i a l bodies. V&V TOOLS AND COMPUTER AIDED SOFTWARE ENGINEERING (CASE) Now, what about t o o l s t o a i d t h e V & V p r o c e s s . Good, s y s t e m a t i c s o f t w a r e development, maintenance and d o c u m e n t a t i o n t e c h n i q u e s a r e o f c o u r s e assumed ( b u t don't l e t them g e t i n t h e way o f p r o g r e s s ) . M a r t i n , O r r , Yourdon and o t h e r s make a good l i v i n g t e a c h i n g and w r i t i n g about t h e s e . I n a d d i t i o n t o t h e environments d e l i v e r e d w i t h ART, KEE, M l / S i and e x p e r t system development s h e l l s s o l d by T I , G o l d H i l l , A r i t y , KDS and o t h e r s , a c o u p l e o f r e c e n t d e m o n s t r a t i o n s have been v e r y i m p r e s s i v e . A I Corp's Knowledge Base Management System (38a) f o r t h e d e s i g n o f systems c o n t a i n i n g knowledge base and e x p e r t system components c o n t a i n s modules t o a i d t h e d e s i g n p r o c e s s w i t h f o r w a r d and backward c h a i n i n g , h y p o t h e t i c a l r e a s o n i n g and o b j e c t o r i e n t e d programming t h a t i n c o r p o r a t e s t h e INTELLECT™ c o n c e p t s o f n a t u r a l language u s e r i n t e r f a c e t o g e n e r a t e w o r k i n g code. Knowledgeware I n c ' s I n f o r m a t i o n E n g i n e e r i n g Workbench (38b) package i s n o t a b l e f o r i t s r e p o r t e d o r i g i n i n 2000 r u l e s and 60,000 l i n e s o f ( A r i t y ) P r o l o g code t o manage s e v e r a l system development m e t h o d o l o g i e s w i t h a l t e r n a t e r e p r e s e n t a t i o n s and l o g i c and c o n s i s t e n c y c h e c k i n g o f i n p u t v i a a s i n g l e shared e n c y c l o p e d i a concept. The i n v o l v e m e n t o f James M a r t i n w i l l g i v e you some i d e a of t h e systems a n a l y s i s , d e s i g n and diagramming m e t h o d o l o g i e s u s e d . The o u t p u t i s C code. P e r c e p t r o n i c s ' Knowledge Shaper (38c) a d v e r t i s e s t h a t i t c o n t a i n s a "knowledge base v a l i d a t i o n and v e r i f i c a t i o n t o o l " . A c o m b i n a t i o n o f t h e s e t h r e e approaches would seem t o g i v e t h e b e s t o f a l l p o s s i b l e ( e x p e r t system development) w o r l d s . Texas I n s t r u m e n t s * P e r s o n a l C o n s u l t a n t and P r o c e d u r e s C o n s u l t a n t (39) s e r i e s o f e x p e r t system development s h e l l s c o u l d a l s o be c o n s i d e r e d a s s p e c i a l i z e d CASE t o o l s . There a r e o f c o u r s e many o t h e r CASE p r o d u c t s and a m u l t i t u d e o f e x p e r t system development " s h e l l s " , l a n g u a g e s and " t o o l s " . Case


134


t o o l s i n c o r p o r a t i n g e x p e r t systems V & V ready market.

technology should f i n d a

CASE STUDY - V&V DURING PROTOTYPE DEVELOPMENT


F a c t o r y M u t u a l e n g i n e e r s and c o n s u l t a n t s a r e i n v o l v e d i n two main a c t i v i t i e s , r i s k a n a l y s i s and l o s s p r e v e n t i o n e n g i n e e r i n g . Our p a r e n t i n s u r a n c e companies ( A l l e n d a l e , A r k w r i g h t , P r o t e c t i o n M u t u a l , F a c t o r y M u t u a l I n t e r n a t i o n a l ) p r o v i d e p r o p e r t y damage and b u s i n e s s i n t e r r u p t i o n i n s u r a n c e c o v e r a g e and r e l a t e d r i s k management s e r v i c e s f o r " h i g h l y p r o t e c t e d r i s k s (HPR)" w o r l d w i d e . The g e n e r i c problem f a c e d by o u r f i e l d p e r s o n n e l i s t h e d e f i n i t i o n ( o r c l a s s i f i c a t i o n ) o f h a z a r d s f o l l o w e d by d e c i s i o n s on a p p r o p r i a t e recommendations f o r c o s t - e f f e c t i v e forms o f b o t h a c t i v e and p a s s i v e protection. Three y e a r s ago we conducted a s u r v e y o f A I Technology and d e c i d e d t h a t d e v e l o p i n g a p r o t o t y p e e x p e r t system would g i v e u s a b e t t e r f e e l f o r b o t h t h e p o t e n t i a l s and t h e problems a s s o c i a t e d w i t h t h i s "new" t e c h n o l o g y . We chose t o f o c u s on t h e c l a s s i f i c a t i o n and p r o t e c t i o n o f o r g a n i c p e r o x i d e s m a i n l y because o f t h e e x i s t e n c e o f FM Loss P r e v e n t i o n Data P u b l i c a t i o n s (48a, 48b) on t h e s u b j e c t and t h e e x i s t e n c e o f some i n - h o u s e e x p e r t i s e . Our e x e r c i s e i n d e f i n i n g t h e h a z a r d c l a s s i f i c a t i o n l o g i c and a p p r o p r i a t e l o s s p r e v e n t i o n measures has r a i s e d a number o f q u e s t i o n s , d i s c l o s e d some p o t e n t i a l problems and i n g e n e r a l g i v e n us a b e t t e r u n d e r s t a n d i n g o f t h e s u b j e c t m a t t e r . T h i s i n i t s e l f has made t h e e x e r c i s e w o r t h w h i l e . A f t e r r e v i e w i n g a number o f A I languages and e x p e r t system development s h e l l s we chose one c a l l e d KDS ( 4 0 ) f o r Knowledge D e l i v e r y Systems f r o m t h e KDS Corp. i n W i l m e t t e , I l l i n o i s t o s e r v e as a development t o o l . The c h o i c e was based on c o n s i d e r a t i o n o f p r i c e / p e r f o r m a n c e , a b i l i t y t o r u n on a PC, ease o f u s e and l a s t l y the d e v e l o p e r , B a r b a r a W a l l a c e , c o n s i s t e n t l y answered t h e phone and f i e l d e d o u r q u e s t i o n s p a t i e n t l y and c o m p e t e n t l y . In addition, the KDS p r o d u c t has seen a s t e a d y stream o f improvements and enhancements w h i c h make i t a v e r y p o w e r f u l t o o l f o r a v e r y c o m p e t i t i v e p r i c e . I t i s w r i t t e n i n Assembly language, w h i c h makes i t f a s t and compact. Our p r o t o t y p e h a z a r d c l a s s i f i c a t i o n system was i n t e n d e d t o be based on t h r e e a p p r o a c h e s , 1) l o o k up t h e answer i n a d a t a b a s e ( e i t h e r o u r s o r o t h e r s ) , 2) i n f e r an answer by c o m p a r i s o n o f u n c l a s s i f i e d m a t e r i a l s w i t h m a t e r i a l s whose c l a s s i s a l r e a d y "known" w i t h some degree o f c e r t a i n t y , o r 3) a s k f o r s u f f i c i e n t i n p u t d a t a ( e i t h e r b a s i c p r o p e r t i e s o r " t e s t " d a t a ) on t h e m a t e r i a l t o a l l o w c a l c u l a t i o n o f t h e p o t e n t i a l h a z a r d from something a k i n t o " f i r s t p r i n c i p l e s : - an a l g o r i t h m i c o r l o g i c a l s o l u t i o n .


10.

MILLER

VenfkaiionandVdidaiionofDeckion Support Expert Systems 135

T h i s approach i s s t i l l c o n s i d e r e d q u i t e v a l i d ; however, o u r e x p e c t a t i o n o f t h e a v a i l a b i l i t y o f unambiguous b a s e l i n e d a t a and w e l l - d e f i n e d , u n i v e r s a l l y - a c c e p t e d c l a s s i f i c a t i o n l o g i c turned out t o be o v e r l y o p t i m i s t i c .


A summary o f some c u r r e n t e f f o r t s i n t h i s a r e a o f m a t e r i a l h a z a r d classification follows: FM/NFPA: The c l a s s i f i c a t i o n p r o c e s s e s o f b o t h FM and NFPA (The N a t i o n a l F i r e P r o t e c t i o n A s s o c i a t i o n ) t u r n e d o u t t o be e s s e n t i a l l y a committee d e c i s i o n based on an undocumented r e v i e w o f m a t e r i a l p r o p e r t i e s and u n v a l i d a t e d t e s t d a t a s u b m i t t e d by m a n u f a c t u r e r s , e i t h e r i n d i v i d u a l l y o r through the auspices o f the Organic P e r o x i d e s P r o d u c e r s S a f e t y D i v i s i o n (OPPSD) o f t h e S o c i e t y o f t h e P l a s t i c s I n d u s t r i e s ( S P I ) . There was and i s t o d a t e no p u b l i s h e d p r o t o c o l d e f i n i n g the l o g i c o f the hazard c l a s s i f i c a t i o n , t e s t i n g and d e c i s i o n making p r o c e s s , a l t h o u g h a s a r e s u l t o f t h i s e f f o r t a l o g i c d i a g r a m has been proposed f o r d i s c u s s i o n by t h e NFPA T e c h n i c a l Committee on t h e S t o r a g e , H a n d l i n g and T r a n s p o r t i o n o f Hazardous C h e m i c a l s . I t i s s t i l l under d i s c u s s i o n . I n t h e meantime t h e r e c o n t i n u e t o be some s i g n i f i c a n t d i f f e r e n c e s between FM and NFPA c o n c l u s i o n s (45, 4 8 a , 48b) r e g a r d i n g b o t h t h e c l a s s i f i c a t i o n and p r o t e c t i o n o f packaged p e r o x i d e f o r m u l a t i o n s . UN/TNO: The UN and TNO ( t h e N e t h e r l a n d s ) a r e grouped t o g e t h e r because o f TNO's major r o l e i n c o o r d i n a t i n g t h e e f f o r t s o f v a r i o u s U.N. o r g a n i z a t i o n s d e a l i n g w i t h t h e t r a n s p o r t o f hazardous m a t e r i a l s . The U.N. Working Group o f E x p e r t s on t h e C l a s s i f i c a t i o n o f O r g a n i c P e r o x i d e s and TNO a r e a l s o i n v o l v e d w i t h r e l a t e d questions o f t h e hazard c l a s s i f i c a t i o n o f o x i d i z e r s . A t present t h e r e seems t o be no b a s i s f o r t h e c o n s t r u c t i o n o f an a n i o n - c a t i o n matrix d e f i n i n g the r e l a t i v e hazard o f a l l i n o r g a n i c o x i d i z e r p o s s i b i l i t i e s , much l e s s one f o r o r g a n i c m a t e r i a l s . The c u r r e n t TNO g u i d e l i n e s f o r s t o r a g e o f o r g a n i c p e r o x i d e s (49) i n c l u d e s c l a s s i f i c a t i o n procedures. F o r t h o s e who w i s h t o keep a b r e a s t o f t h e a c t i v i t i e s o f n a t i o n a l and i n t e r n a t i o n a l groups d e a l i n g w i t h t h e t r a n s p o r t o f hazardous m a t e r i a l s , t h e "Hazardous Cargo B u l l e t i n " , p u b l i s h e d i n t h e UK, has t i m e l y t e c h n i c a l updates. The paper by P i l z ( 4 ) p r e v i o u s l y mentioned i n d i c a t e s t h a t t h e s e t r a n s p o r t a t i o n h a z a r d c l a s s e s a r e a l s o b e i n g used t o d e t e r m i n e warehousing requirements, although t r a n s p o r t hazard t e s t procedures as p r e s e n t l y conducted do n o t a d e q u a t e l y a d d r e s s t h e s p r i n k l e r system d e s i g n and d e n s i t y r e q u i r e m e n t s f o r l a r g e q u a n t i t i e s o f h a z a r d o u s m a t e r i a l s i n a v a r i e t y o f p a c k a g i n g and s t o r a g e c o n f i g u r a t i o n s . I t i s , however, a s i g n i f i c a n t f i r s t s t e p toward m i n i m i z i n g t h e p o t e n t i a l f o r a S c h w e i z e r h a l l e - t y p e i n c i d e n t . The p r e s e n t U.N. c l a s s i f i c a t i o n l o g i c diagram f o r o r g a n i c p e r o x i d e s a s p u b l i s h e d i n t h e l a t e s t U.N. "Orange Book" i s shown i n F i g u r e s 2 and 3. The r e l a t i o n s h i p between t h e f l o w c h a r t " e x i t " (A-G) and the U.N. P a c k i n g (Hazard) Groups ( I - I I I ) i s s t i l l somewhat e l u s i v e .



136


Test Series

TA TB TC TD TE TF TG

Detonation Detonation in package Deflagration Deflagration in package Heating under confinementD e c i s i o n Y Yes Explosive power N No Explosion in package P Partial R Rapid S Slow Exit

A B C D E F G T:

V M L >L Y,P

Violent Medium Low Greater than low Yes or partial

Should not be accepted for transport in that package Accept in packages < 25 kg net mass, special provisions 181 Accept in packages < 50 kg net mass, special provision 180 Accept in packages < 50 kg net mass, special provision 180 Accept in packages < 400 kg/450 L May be considered for bulk shipment May be considered for exception (see 11.3.2.2 (H))

Figure 3. Key to U.N. organic peroxides test protocol. (Adapted from ref. 46.)


10.

MILLER

Verification and VdidaHon ofDecision Support Expert Systems 137


Work on p e r o x i d e s i s c u r r e n t l y b e i n g c o o r d i n a t e d by TNO; a d a t a b a s e o f t e s t i n f o r m a t i o n from l a b o r a t o r i e s w o r l d w i d e has been d e v e l o p e d and t h e c l a s s i f i c a t i o n p r o t o c o l i s b e i n g r e f i n e d . The n e x t s t e p i s t o reduce t h e number o f "Orange Book" e n t r i e s f o r o r g a n i c p e r o x i d e s from o v e r a hundred t o twenty g e n e r i c ( c l a s s ) e n t r i e s . H o p e f u l l y the w o r k i n g groups w i l l a l s o g i v e some c o n s i d e r a t o n t o w a r e h o u s i n g and warehouse p r o t e c t i o n r e q u i r e m e n t s t o a c c o u n t f o r problems a t b o t h ends o f t h e t r a n s p o r t p r o c e s s . The h a z a r d s d u r i n g m a n u f a c t u r e and i n - p r o c e s s use o f p e r o x i d e s a r e a n o t h e r m a t t e r e n t i r e l y . It is a n t i c i p a t e d t h a t more c o n s i d e r a t i o n w i l l a l s o be g i v e n t o t h e arrangement and p r o t e c t i o n o f hazardous m a t e r i a l s d u r i n g t r a n s p o r t as w e l l a s q u a n t i t y and c o m p a t i b i l i t y l i m i t a t i o n s . ASTM: S a u l P a t a i ' s book "The C h e m i s t r y o f P e r o x i d e s " (41) was r e v i e w e d w i t h some hope o f f i n d i n g e i t h e r a s y s t e m a t i c e v a l u a t i o n of r e a c t i v i t y as i t r e l a t e s t o the p o t e n t i a l hazard o f peroxides o r some b a s i c c o n c e p t s t h a t would a l l o w f o r m u l a t i o n o f a u n i f y i n g t h e o r y t h a t would p e r m i t an a l g o r i t h m i c s o l u t i o n t o t h e c l a s s i f i c a t i o n problem. U n f o r t u n a t e l y , when S a u l g o t t o t h e c h a p t e r on s a f e t y he acknowledges t h a t he gave up. B r e t h e r i c k ' s "Handbook o f R e a c t i v e C h e m i c a l H a z a r d s " ( 4 2 ) p r e s e n t s some u s e f u l i n f o r m a t i o n and i n c i d e n t h i s t o r i e s , b u t p r o v i d e s l i t t l e i n t h e way o f c o o r d i n a t e d i n s i g h t based on m o l e c u l a r c o m p o s i t i o n and s t r u c t u r e t o a l l o w f o r s y s t e m a t i c e x t r a p o l a t i o n t o new m a t e r i a l s . D i s c u s s i o n s have a l s o been h e l d w i t h ASTM E27 Committee (On The Hazard P o t e n t i a l o f C h e m i c a l s ) members r e g a r d i n g t h e use o f t h e CHETAH ( C h e m i c a l Thermodynamics and Hazards E s t i m a t i o n Program) software t o a i d i n the c l a s s i f i c a t i o n process. Work i s p r o c e e d i n g (by E27.07 E s t i m a t i o n Methods) on b o t h a PC v e r s i o n o f t h e e x i s t i n g CHETAH program ( w i t h c o n s i d e r a b l e u p d a t i n g ) and a new mainframe (and p o s s i b l e PC) v e r s i o n w i t h expanded c a p a b i l i t i e s . CHETAH i s d e s c r i b e d i n d e t a i l i n r e f e r e n c e s 4 3 a , b, c. C o p i e s o f t h e TNO p e r o x i d e t e s t d a t a b a s e s have been p r o v i d e d t o E27.07 and t h e new v e r s i o n s o f CHETAH a r e e x p e c t e d t o c o n t a i n an e x t e n s i v e database as w e l l as p a t t e r n - r e c o g n i t i o n techniques f o r e s t i m a t i n g t h e h a z a r d o f new m a t e r i a l s . The CHETAH s o f t w a r e w i l l c o n t i n u e t o r e l y on bond energy d a t a and group c o n t r i b u t i o n c a l c u l a t i o n s t o e s t i m a t e energy r e l e a s e p o t e n t i a l . H o p e f u l l y , t h e new v e r s i o n s w i l l a l s o i n c o r p o r a t e n a t u r a l language e x p e r t system-type f r o n t ends so t h a t t h e CHETAH program(s) w i l l see expanded use i n b o t h a n a l y t i c a l and t u t o r i a l modes. C o p i e s o f t h e L E I L A ( 8 ) d i s s e r t a t i o n have a l s o been p r o v i d e d t o E27.07 a s an example o f an e x p e r t system a p p r o a c h t o s e l e c t i o n and use o f a p p r o p r i a t e t h e o r i e s and c o m p u t a t i o n a l methods f o r t h e s o l u t i o n o f problems i n c h e m i c a l k i n e t i c s . The ASTM Committee E27.02 (Thermal S t a b i l i t y / C o n d e n s e d Phases) has a l s o r e c e n t l y p u b l i s h e d a document ( 4 4 ) e n t i t l e d " S t a n d a r d P r a c t i c e f o r C a l c u l a t i o n o f Hazard P o t e n t i a l F i g u r e s o f M e r i t f o r T h e r m a l l y U n s t a b l e M a t e r i a l s " . T h i s i n c l u d e s c a l c u l a t i o n s o f 1) Time t o t h e r m a l runaway, 2) c r i t i c a l h a l f t h i c k n e s s , 3) c r i t i c a l t e m p e r a t u r e , and 4) a d i a b a t i c d e c o m p o s i t i o n t e m p e r a t u r e r i s e .


138


U n f o r t u n a t e l y , o r f o r t u n a t e l y depending on your p e r s p e c t i v e , t h e k i n e t i c parameters needed i n t h i s c a l c u l a t i o n a r e o b t a i n e d by D i f f e r e n t i a l S c a n n i n g C a l o r m e t r y (DSC) and t o my knowledge no s y s t e m a t i c a l l y - a c q u i r e d d a t a has been o b t a i n e d on a l a r g e number o f pure p e r o x i d e s and c o m m e r c i a l f o r m u l a t i o n s and t h e i r v a r i a t i o n s . The e x e r c i s e c o u l d have b o t h p r a c t i c a l and t h e o r e t i c a l b e n e f i t s .


SUMMARY-STATUS OF RESEARCH BY THE UN, TNO, ASTM, FMRC In summary, work r e l a t i n g t o v a r i o u s approaches t o t h e c l a s s i f i c a t i o n o f hazardous m a t e r i a l s i s c u r r e n t l y underway w i t h i n committees o f t h e NFPA ( 4 5 ) , ASTM ( 4 3 a , b, c, 44) and t h e UN ( 4 6 ) . The TNO i n H o l l a n d i s s p e a r h e a d i n g much o f t h e work on p e r o x i d e s and o x i d i z e r s . I t appears t h a t a l l o f t h e s e groups w i l l produce s i g n i f i c a n t new r e s u l t s by t h e end o f 1988. I n t h e meantime, t h e F a c t o r y M u t u a l R e s e a r c h C o r p o r a t i o n (FMRC) has been making s i g n i f i c a n t p r o g r e s s i n d e v e l o p i n g t e s t methods f o r c l a s s i f y i n g t h e h a z a r d o f b o t h m a t e r i a l s and commodities. One o f t h e s e programs i s summarized i n a paper e n t i t l e d " G e n e r a t i o n o f Heat and C h e m i c a l Compounds i n F i r e s " by Dr. A. Tewarson (47) w h i c h appears a s a c h a p t e r i n t h e new SFPE ( S o c i e t y o f F i r e P r o t e c t i o n E n g i n e e r s ) Handbook p u b l i s h e d t h i s f a l l . CONCLUSIONS Because o f c o s t and time c o n s t r a i n t s , we have n o t had t h e o p p o r t u n i t y t o a c t u a l l y work w i t h and e v a l u a t e many o f t h e t o o l s mentioned. C o n s e q u e n t l y o u r c o n c l u s i o n s a r e based i n l a r g e measure on t h e comments r e c e i v e d from t h o s e w o r k i n g i n t h e f i e l d s o f A I / E x p e r t Systems and S o f t w a r e V e r i f i c a t i o n and V a l i d a t i o n . Also, my c o n c l u s i o n s a r e t h o s e o f a p o t e n t i a l u s e r r a t h e r t h a n o f a s p e c i a l i s t i n the f i e l d . The p r i n c i p l e c o n c l u s i o n i s t h a t t h e development o f " e x p e r t " systems and t h e n e c e s s a r y a u d i t t o o l s f o r v e r i f i c a t i o n and v a l i d a t i o n o f t h o s e systems i s s t i l l i n i n f a n c y . There a r e a number o f i n t e r e s t i n g c o n c e p t s under development w h i c h bear p r o m i s e but w h i c h w i l l p r o b a b l y see t h e i r f i r s t a p p l i c a t i o n w i t h i n c r i t i c a l space and d e f e n s e a p p l i c a t i o n s . T o o l s u s e f u l t o t h o s e o f us i n t e r e s t e d i n more g e n e r a l i z e d r i s k and r e l i a b i l i t y a n a l y s e s a r e p r o b a b l y a t l e a s t two t o t h r e e y e a r s away. Mike Brown, chairman o f t h e DOD T r i - S e r v i c e S o f t w a r e Systems S a f e t y Working Group (SSSWG), emphasizes t h a t " S a f e t y must be d e s i g n e d i n t o a system and p a r t i c u l a r l y t h e s o f t w a r e a t t h e e a r l i e s t p o s s i b l e s t a g e o f development." "We c a n ' t V&V a r e q u i r e m e n t t h a t i s n ' t t h e r e o r doesn't e x i s t . " "The most common problem t h a t we e n c o u n t e r i n s o f t w a r e systems s a f e t y i s i n c o m p l e t e s p e c i f i c a t i o n s and i n a d e q u a t e d e s i g n from t h e s a f e t y p e r s p e c t i v e . " The T r i - S e r v i c e Working Group i s c u r r e n t l y w o r k i n g t o produce a "how t o " handbook t o augment o r supersede c u r r e n t g u i d e l i n e s . A g a i n , good s o f t w a r e e n g i n e e r i n g p r a c t i c e s h o u l d i n c l u d e s a f e t y c o n s i d e r a t i o n s a t t h e e a r l i e s t s t a g e and throughout t h e l i f e c y c l e .


10.

MILLER

Verification and Validation ofDecision Support Expert Systems 139

And s o f t w a r e , e x p e r t systems i n c l u d e d , s h o u l d be d e s i g n e d tested.

t o be


E x p e r t systems t e c h n o l o g y c a n be u s e f u l . We have demonstrated t h e ease o f use and l o g i c - d o c u m e n t a t i o n c a p a b i l i t i e s o f a p r o t o t y p e u s i n g t h e UN f l o w c h a r t and t h e KDS e x p e r t system development s o f t w a r e . KDS Corp p e r s o n n e l have a l s o demonstrated t h e r e l a t i v e ease w i t h w h i c h o u r Loss P r e v e n t i o n Data Sheets c a n be r e p r e s e n t e d i n e x p e r t systems f o r m a t . They w i l l be happy t o e x p l a i n t h e i r demo "PEROX" based on o u r D.S. 7-81 ( 4 8 ) . A t p r e s e n t , however, t h e "PEROX" demo has been n e i t h e r funded n o r endorsed by t h e FM System. P e r o x i d e h a z a r d c l a s s i f i c a t i o n e x p e r t systems development a t FM/Norwood i s p r e s e n t l y on h o l d u n t i l some o f t h e r e s o u r c e s noted become a v a i l a b l e . We have a way t o go b e f o r e a f u l l y v a l i d a t e d c l a s s i f i c a t i o n model i s c o m p l e t e . A t t h e same t i m e , we a r e c o n t i n u i n g t o e x p l o r e the p o s s i b i l i t y o f u s i n g chemical database programs such a s t h o s e a v a i l a b l e f r o m M o l e c u l a r D e s i g n I n c . and t h e U n i v e r s i t y o f S a n t i a g o , C h i l e (ARIUSA) a s components o f o u r c h e m i c a l h a z a r d e x p e r t s y s t e m s . We a r e a l s o l o o k i n g a t c h e m i c a l d a t a b a s e s on o p t i c a l d i s c such a s t h o s e a v a i l a b l e from DuPont, A l d r i c h , M i c r o m e d i x and t h e Canadian C e n t e r f o r O c c u p a t i o n a l S a f e t y and H e a l t h a s components o f f u t u r e systems. THE FUTURE Combinations o f p o r t a b l e microcomputers, o p t i c a l d i s c / c a r d knowledge b a s e s , e x p e r t systems t e c h n o l o g y , n a t u r a l language p r o c e s s i n g and machine t r a n s l a t i o n w i l l have a major impact on t h e way we do b u s i n e s s i n t h e f u t u r e . The a c c e p t a n c e o f e x p e r t systems i n p r o c e s s c o n t r o l a p p l i c a t i o n s and i n r i s k a s s e s s m e n t , r i s k management and l o s s p r e v e n t i o n e n g i n e e r i n g w i l l depend t o a l a r g e e x t e n t on t h e degree t o w h i c h v e r i f i c a t i o n and v a l i d a t i o n q u e s t i o n s a r e a d d r e s s e d e a r l y on i n t h e d e s i g n o f t h e systems.

References 1

Anon "Programmable Controllers" Factory Mutual Loss Prevention Data 5-27/14-27 (Feb 1988).

2a

Shum, S. K . , Davis J.F., Punch, W.F. and Chaudrasekaran, 13 "An Expert Systems Approach to Malfunction Diagnosis i n Chemical Plants". Computers i n Chemical Engineering, Vol. 12, No. 1, 1988, pp 27-36.

2b Klopman, Gilles and Raychaudhury, Chandan, "A Novel Approach To The Use of Graph Theory in Structure-Activity Relationship Studies. Application to the Qualitative Evaluation of Mutagenicity i n a Series of Nonfused Ring Aromatic Compounds." Journal of Computational Chemistry, April/May 1988



140


3

Wu, Yun-Fu and Lidsky, L.M. "Nuclear Power Plant Technical Specifications Computerization: Application of Prolog" MIT-NPI-TR-008, May 86.

4

P i l z , Volker (Bayer),"Sichere Lagerung von Stoffen i n der chemischen Industrie". Chemie Ingenieur Technik, 60 (1988) No. 6, S.456-463.

5

Anon. Various "German Workshops on Artificial Intelligence (GWAI) published annually by Springer Verlag as part of the Fachberichte Informatik Series

6

Anon. "Proceedings of The Second Annual Artificial Intelligence Research Forum" NASA/Ames, April 1987.

7

Anon. "Proceedings of the First Annual Workshop on Space Operations Automation and Robotics (SOAR-87) NASA/Johnson Space Center, Houston, TX, August 1987.

8

Sauk, Brian Edward, "LEILA: An Expert System for Estimating Chemical Reaction Rates" PhD Dissertation, Carnegie Mellon Univ., A p r i l , 1987.

9

Nicholosi, Stephen L. (Battelle-Columbus) "SOPHIE: An Expert System for Selection of Hazard Evaluation Procedures" 99 pp + disc, Oct. 28, 1987

10 Anon "Guidelines for Hazard Evaluation Procedures" prepared by Battelle-Columbus for the AIChE Center for Chemical Process Safety (CCPS), 1985 11 Anon "System Safety program Requirements: MIL-STD-88213 (30 March 1984) 12 Anon "Software Systems Safety" MIL-STD-882B-Notice 1 (3/84) 13 Anon-Software Systems Safety handbook: AFISC SSH-1-1 (5 September 1985) (U.S. A i r Force) 14 Anon "Software Systems Safety" Hazard Prevention-Journal of the Systems Safety Society, V o l . 22, Nos. 1, 2, 3, 6, 1986 15 Schoitsch, Erwin "Software Safety and Software Quality Assurance i n Real Time Applications". Computer Physics Communications, 50 (1988) 169-211. 16 Luck, Heinz O. and Schlossarek, Udo "Software Controlled Fire Detection Systems". Fire Science and Technology, Vo. 7, Nov. 2, 1987, pp 53.60. 17a Anon "The Engineers Responsibility for Computer Based Decisions". Published by the Institution of Chemical Engineers, UK, 1987.


10. MILLER

Verification and Validation ofDecision Support Expert Systems 141

17b Anon "Chemical Process Control and Control Rooms" Factory Mutual Loss Prevention Data Publication 7-45 (7/74) 17c Anon "Chemical Recovery Boilers" Factory Mutual Loss Prevention Data Publication 6-21/12-21 (9/85). 18 Lipow, Myron and Book, Erwin "Implication of R&M 2000 on Software" IEEE Transactions on R e l i a b i l i t y , Vol. R-36, No. 3, August 1987, pp. 355-361


19 Anon "Intelligent Data/Knowledge Base System" Sol F30602-88-R-0071, February 1988, Rome A i r Development Center, USAF 20a Mill, Ali "An Introduction to Formal Program Verification" VNR 1985 20b Quirk, W.J. "Verification and Validation of Real Time Software", Springer Verlag, 1985 20c DeMillo, Richard A . , McCracken, W. Michael, Martin, R.J., Passafiume, John F. "Software Testing and Evaluation" Software Engineering Research Center. Georgia institute of Technology, Benjamin/Cummings, 1987 21 Wilson, Dean K. "Failure Mode Management: A Loss Prevention Philosophy for Programmable Logic Controllers" AIChE Loss Prevention Symposium, Mar 1988. 22 McMillan, Gregory K. "Can You Say 'Process Interlocks'?" Intech Journal, May 1987. Presented at the AIChE Loss Prevention Symposium, Mar 1988. 23 Thomas, Harold W. "A Quantitative Approach To The Use of Programmable Controllers i n Safety Circuits". AIChE Loss Prevention Symposium Mar 1988. 24 Anon "PES's and the HSE Guidelines "The Chemical Engineer, Loss Prevention Supplement, No. 439, August 1987, pp. 16-17. 25 Bryan, L.A. and Bryan, E.A. "Programmable Controllers - Theory and Implementation" Industrial Text, 1988. 26 Politakis, Peter "Empirical Analysis for Expert Systems" PhD Thesis, Rutgers, 1985. 27 Politakis, P. and Weiss, S. "Using Empirical Analysis to Refine Expert System Knowledge Bases" Artificial Intelligence Journal 22 (1984) 23-48. 28 Geissman, James R. and Schultz, Roger D. "Verification and Validation of Expert Systems" AI Expert, February 1988, pp. 26-33.


142


29a Stachowitz, R . A . , Chang, C.L. and Combs J . B . "Performance Evaluation of Knowledge Based System" Proceedings of the Third Aerospace Applications of Artificial Intelligence" Conference (AAAIC), Dayton, Ohio, Oct. 5-9, 1987 (Keynote Address). 29b Stachowitz, R . A . , Chang, C . L . , Stock, T.S. and Combs, J . B . "Building Validation Tools for Knowledge-Based Systems" Proceedings of the First Annual Workshop on Space Operations Automation and Robotics (SOAR), Houston, TX, Aug. 1987, NASA Conf. Pub. 2492 pp. 209-216.


29c Stachowitz, R. A . , Combs, J . B . and Chang, C.L. "Validation of Knowledge-Based Systems:, Proceedings AIAA/NASA/USAF Symposium on Automation, Robotics and Advanced Computing, Arlington, VA, Mar. 1987 29d Stachowitz, A.H. Combs, J . B . "Validation of Expert Systems" Proceedings Twentieth Hawaii International Conference on System Siences, Kona, Hawaii, Jan 1987 30 Goodwin, Mary Ann and Robertson, Charles C. (Rockwell) "Expert System Verification Concerns in an Operating Environment NASA SOAR Conference, August 1987, pp. 203-207 31 Castore, Glen (Honeywell) "A Formal Approach to Validation and Verification for Knowledge Based Control Systems". NASA/JSC SOAR Conference, August 1987, pp. 197-202. 32 Culbert, Chris, Riley, Gary and Savely, Robert T. (NASA) "Approaches to The Verification and Validation of Rule Based Expert Systems" NASA/JSC SOAR Conference, August 1987, pp. 191-196a. 33 Shapiro, E. "Algorithmic Program Debugging - Using Prolog" MIT Press. 34 Covington, Michael A . , Nute, Donald, Vellino, Andre, "Prolog Programming i n Depth" Scott, Foresman, 1988 35 Kanamori, Tadashi, et a l (Mitsubishi) "Argus/V; A System for Verification of Prolog Programs" ICOT report TR-176, May 1986

36 Richardson, Keith and Wong, Carla, "Knowledge Based Systems Verification and Validation" Proceedings of the Second Annual Artificial Intelligence Research Forum, April 1987 NASA/Ames. 37 Groundwater, E.H. et a l (Science Applications) "Approaches to the Verification and Validation of Expert Systems for Nuclear Power Plants" Electric Power Research Institute, EPRI NP 5236, July 1987 38a AI Corp - "Knowledge Base Management System"

617-890-8400

38b Knowledgeware, Inc. - "Information Engineering Workbench" 404-231-8575


10. MILLER

Verification and Validation ofDecision Support Exw

38c Perceptics (Westinghouse) AI Applications Division "Knowledgeshaper" - 615-966-9200 39 Texas Instruments "Personal Consultant Series" 1-800-527-3500 or i n Europe (44) + (234) + 22 41 87 40 KDS Corp "Knowledge Delivery System - KDS 3.0" 312-251-2621 41 Patai, Saul "The Chemistry of Peroxides".

Wiley, 1983


42 Bretherick, L . "Handbook of Reactive Chemical Hazards", Third Edition, 1985. Butterworths, Stoneham, MA 43a Davies, C.A. (Dow), Kipnis, I.M. (FMC), Chase, M.W. (Dow) and Treweek, D. N (consultant/Battelle), "Estimation of Thermochemical and Hazard Data of Chemicals Using the ASTM CHETAH (Chemical Thermodynamics and (Energy Release) Hazard Evaluation). Proceedings of the spring 1984 meeting of ACS i n St. Louis. ACS Symposium Series 274 (Chemical Process Hazard Review) PP 81-90 (1985) 43b Yoshida, Tadao "Safety of Reactive Chemicals" Industrial Safety Series V o l . 1, Elsevier, 1987 43c Ducros, M and Sannier H. "Application du Programme Chetah a L'Etude de la Sensibilisation de Composes Oxygenes et a l'Estimation des Limites Inferieures d'Inflammabilite", Journal of Hazardous Materials, 19 (1988) 33-49 44 Anon "Standard Practice for Calculation of Hazard Potential Figures of Merit for Thermally Unstable Materials", ASTM, 1987 Standard E 1231 45 Anon "Standard for Storage of Organic Peroxides" NFPA, 1986 46 Anon "Recommendations on the Transport of Dangerous Goods Tests and Criteria". First edition of Addendum I, Part III "Tests and Criteria for the Classification of Organic Peroxides." United Nations Publication 47 Tewarson, A. "Generation of Heat and Chemical Compounds i n Fires". Factory Mutual Research Corporation, J.I. 0N1E1.RC (NS), March 1987 48a Anon "Organic Peroxides". Factory Mutual System Loss Prevention Data 7-80, March 1972. 48b Anon, "Organic Peroxides Hazard Classification (tentative)" Factory Mutual System, Loss Prevention Data 7-81, August 1986. 49 deKoning, J. et a l "Storage of Organic Peroxides" CPR 3E, Committee for The Prevention of Disasters Caused by Dangerous Substances (CPR), First Edition 1982, from the Labour Inspectorate, The Netherlands


143

144

EXPERT SYSTEM APPLICATIONS IN CHEMISTRY APPENDIX AIChE MEETING A I PAPERS


A p r i l 1986 - New O r l e a n s An E x p e r t System Approach t o On-Line O p t i m i z a t i o n An E x p e r t System f o r P r o c e s s C o n t r o l T o u b l e s h o o t i n g ( P r o l o g ) O b j e c t - O r i e n t e d I n t e g r a t i o n i n P r o c e s s E n g i n e e r i n g Computation Dynamic S i m u l a t i o n i n t h e C o n s t r u c t i o n o f E x p e r t Systems f o r Process F a u l t Diagnosis PROVIAL - A T o o l f o r B a t c h P r o c e s s D e s i g n and E v a l u a t i o n November 1986 - Miami Beach A Computer-Aided L a b o r a t o r y F o r A r t i f i c i a l I n t e l l i g e n c e A p p l i c a t i o n s i n C h e m i c a l and B i o c h e m i c a l E n g i n e e r i n g Chemical Network Problems S o l v e d on NASA/Goddard*s M a s s i v e l y P a r a l l e l P r o c e s s o r (MPP) Computer Symbolic Computation and C h e m i c a l E n g i n e e r i n g Model Development An O v e r a l l S t r a t e g y f o r t h e S y s t e m a t i c S y n t h e s i s and O p t i m i z a t i o n o f L a r g e - S c a l e Chemical P r o c e s s i n g Systems Synthesis of S o l i d s Processing (Heuristics & Selection) The A p p l i c a t i o n o f Automata and Formal Language Theory t o Chemical Process C o n t r o l S y s t e m a t i c A n a l y s i s o f C h r o m a t o g r a h i c System f o r P r o t e i n P u r i f i c a t i o n (Knowledge-Based) A Fuzzy E x p e r t System f o r S e p a r a t i o n Sequencing o f B i o p r o d u c t s Computer A i d e d M o d e l i n g o f B a c t e r i a C e l l s : The Use o f E x p e r t Systems P r e d i c t i o n o f t h e Dynamic B e h a v i o r o f Temperature S e n s i t i v e Recombinent C e l l s U s i n g a H y b r i d H e u r i s t i c s and M a t h e m a t i c a l Model Based E x p e r t System An E x p e r t Approach Towards S t a t e E s t i m a t i o n o f B i o r e a c t o r s Stop R e i n v e n t i n g P i l o t P l a n t C o n t r o l Systems F l e x i b l e B a t c h P r o c e s s i n g C o n t r o l System ( L o g i c B l o c k ) A r t i f i c i a l I n t e l l i g e n c e - Emerging S c i e n c e f o r C o n t r o l s Systems Applications S e l f L e a r n i n g A d v i s o r y Technology On t h e D e s i g n o f a B l o c k Diagram Based S i g n a l P r o c e s s i n g and S i m u l a t i o n Package Mar/Apr 1987 - Houston An I n t e l l i g e n t System f o r t h e D e s i g n o f P l a n t - W i d e C o n t r o l Configuration D e s i g n K i t : An I n t e l l i g e n t I n t e r f a c e and Database f o r P r o c e s s Engineering The Development o f ( P I P E x p e r t System) a n I n t e r a c t i v e User I n t e r f a c e f o r Chemical Flowsheet S y n t h e s i s The F a l c o n P r o j e c t : An A p p l i c a t i o n o f a n E x p e r t System t o F a u l t Diagnosis Narrowing D i a g n o s t i c Focus by C o n t r o l System D e c o m p o s i t i o n Model-Based R e a s o n i n g f o r F a u l t D i a g n o s i s An E x p e r t System Approach t o D i a g n o s i s o f P r o d u c t Q u a l i t y Deviations PRISIM - An E x p e r t System f o r P r o c e s s R i s k Management A n a l y z i n g P r o c e s s Alarms When Your E x p e r t System F a i l s


10.

MILLER

Verification andValidation oj Decision Support Expert Systems 145

Disturbance A n a l y s i s Using Digraphs Real-Time D i a g n o s i s o f C h e m i c a l P r o c e s s e s : A Model Base A l g o r i t h m O p t i m a l L o g i c f o r M u l t i - C h a n n e l P r o t e c t i v e Systems D u r i n g On-Line Maintenance


I n t e g r a t e d R i s k Assessment Program f o r R i s k Management August 87 - M i n n e a p o l i s The Use o f A r t i f i c i a l I n t e l l i g e n c e i n D i s t r i b u t e d C o n t r o l Model Based Reasoning A p p r o a c h t o C h e m i c a l P l a n t D e s i g n Use o f E x p e r t Systems i n N u c l e a r Power P l a n t s A p p l i c a t i o n o f A I t o Management and A n a l y s i s Problems November 1987 - New York H e u r i s t i c M a n i p u l a t i o n o f P r o c e s s I d e n t i f i c a t i o n and A d a p t i v e Control Algorithms POPS: The P r o t o t y p e O p e r a t i n g Procedure S y n t h e s i s Program Expert M u l t i v a r i a b l e C o n t r o l An O p e r a t o r A d v i s o r f o r C o n t r o l l i n g C o r r o s i o n i n a Crude Fractionator COMA: A C o n f i g u r a b l e O p e r a t o r M o n i t o r and A d v i s o r I n t e g r a t e d i n t o a R e a l Time C o n t r o l System Knowledge-Based R e a l Time Sensor I n t e r p r e t a t i o n f o r P r o c e s s P l a n t s O p e r a t i o n I n s t r u c t i o n System Q u a l i t a t i v e M o d e l i n g o f Dynamic Systems Heat Exchanger Network S y n t h e s i s : A Knowledge E n g i n e e r i n g Approach An E x p e r t System f o r D e s i g n i n g D i s t i l l a t i o n Column P l a t e s STES: A S e p a r a t i o n P r o c e s s e s E x p e r t System RIP - A P r o t o t y p e E x p e r t System f o r R e t r o f i t t i n g C h e m i c a l P l a n t s D e s i g n o f Polymer Composites: A B l a c k b o a r d Approach POPS: The P r o t o t y p e O p e r a t i n g P r o c e d u r e S y n t h e s i s Program O p e r a t o r - A s s i s t e d L e a r n i n g i n E x p e r t Systems f o r F a u l t D i a g n o s i s A C o n n e c t i o n i s t E x p e r t System Approach t o F a u l t D i a g n o s i s i n the Presence o f N o i s e and Redundancy E x p e r t System i n a Wastwater Treatment P r o c e s s D i a g n o s i s An O p e r a t o r A i d f o r A n a l y s i s o f D i s t u r b a n c e s i n D i s t i l l a t i o n Columns M0LD0CT0R: An E x p e r t System f o r F a u l t D i a g n o s i s and Remedies i n I n j e c t i o n Molding of P l a s t i c s A Method o f F a u l t D i a g n o s i s : P r e s e n t a t i o n o f a Deep-Knowledge System R e a l Time Hazard A v e r s i o n and F a u l t D e t e c t i o n : M u l t i p l e Loop C o n t r o l Example Overview of A r t i f i c i a l I n t e l l i g e n c e The E x p e r t System f o r C h e m i s t r y : P l a n n i n g M u l t i s t e p Syntheses and C o n v e r s i o n o f P o l y m e r i c and Hazardous C h e m i c a l Wastes t o Useful Materials The Use o f E x p e r t Systems i n N e g a t i v e M a n u f a c t u r i n g Q u a l i t a t i v e S i m u l a t i o n o f Continuous C h e m i c a l P r o c e s s e s The Use o f E x p e r t Systems i n P r o c e s s D e s i g n o f O f f s h o r e O i l and Gas P r o d u c t i o n Systems A p p l i c a t i o n o f E x p e r t Systems t o E n v i r o n m e n t a l Problems C e l l u l a r Metabolism: An A p p r o a c h Through A r t i f i c a l I n t e l l i g e n c e An I n t e l l i g e n t System f o r the D e s i g n o f B i o c h e m i c a l Pathways


146



E x p e r t Systems, M e t a b o l i s m and P h a r m a c o k i n e t i c s S t r a t e Gene, An E x p e r t System f o r Use i n M o l e c u l a r B i o l o g y An E x p e r t System Based I n t e l l i g e n t C o n t r o l Scheme f o r B i o r e a c t o r s An I n t e l l i g e n t System f o r t h e D e s i g n o f P r o t e i n Recovery P r o c e s s e s March 1988 - New O r l e a n s S y m b o l i c Computing i n P r o c e s s F l o w s h e e t i n g I m p l e m e n t a t i o n o f an E x p e r t System f o r On-Line F a u l t D i a g n o s i s i n a Commercial S c a l e C h e m i c a l P r o c e s s Impact and Requirements o f Knowledge-Based S o f t w a r e E x p e r t Systems i n t h e P u l p and Paper I n d u s t r y I C I P r o c e s s E n g i n e e r i n g E x p e r t Systems - P r e s e n t and F u t u r e Development o f a D i a g n o s t i c E x p e r t System f o r a Whipped Toppings Process E x p e r i e n c e U s i n g an A l Language and an E x p e r t System S h e l l f o r D e v e l o p i n g "BUS1" Process Scheduling Using A r t i f i c i a l I n t e l l i g e n c e To A I o r n o t t o A I , That i s t h e Q u e s t i o n E x p e r t Systems f o r C o n t r o l o f P l a n t O p e r a t i o n s i n R e a l Time U s i n g E x p e r t Systems t o M o n i t o r S a f e t y M a r g i n s f o r H i g h Hazard Processes A N a t u r a l Language I n t e r f a c e t o R e l a t i o n a l Databases f o r Waste Management I n t e l l i g e n t Computer-Assisted

I n s t r u c t i o n Programs

August 1988 - Denver A r t i f i c i a l I n t e l l i g e n c e a t Wang L a b o r a t o r i e s A N a t u r a l Language I n t e r f a c e t o R e l a t i o n a l Databases f o r Waste Management E x p e r t Systems "To Go": Laptop P e r s o n a l Computers & KB S o f t w a r e An A p p l i c a t i o n o f O b j e c t O r i e n t e d Programming To P r o c e s s Simulation Hands-On D e m o n s t r a t i o n o f A I Programs R E C E I V E D June 9,1989


Verification and Validation of Decision Support Expert Systems - ACS

Recommend Documents