Information
•
Textbooks
•
Media
•
Resources
EPOCH: An Organic Chemistry Homework Program That Offers Response-Specific Feedback to Students Raghu Ram Chamala, Roxana Ciochina, and Robert B. Grossman* Department of Chemistry, University of Kentucky, Lexington, KY 40506-0055; *
[email protected] Raphael A. Finkel, Saravana Kannan, and Prasanth Ramachandran Department of Computer Science, University of Kentucky, Lexington, KY 40506-0495
In the typical introductory organic chemistry course, students work questions out of a textbook for which a solutions manual is available. They struggle with a question for a few minutes, look up the answer, and then believe they know how to answer the question. It isn’t until exam time that they realize that knowing an answer to a question and knowing how to answer a question are two entirely different propositions. Computer programs that tell a student that a response is incorrect without supplying the correct answer are likely to be more effective teaching tools. Many Web-based chemistry homework programs exist (1–5), but most require text-based, numeric, or multiple-choice answers. In organic chemistry, however, answers are usually structural drawings. The only program we are aware of that allows a student to draw a chemical structure in response to a question is OWL1 (1), as implemented at the Web site for McMurry’s Organic Chemistry (6). The electronic program for organic chemistry homework (EPOCH) is a Web-based application designed specifically for instructors and students of introductory organic chemistry. EPOCH prompts students to construct a structural response to a posed question with a graphical structure-drawing interface. EPOCH offers feedback that explains why a response is correct or incorrect, guiding students to the correct answer. The author of a question decides what characteristics of a student’s response should elicit what feedback from EPOCH: for example, the presence of a particular atom or functional group (“Br is a leaving group; it should not be present in your product.”), the absence of a certain skeleton (“No C–C bonds should be formed or broken in this reaction.”), or the configuration of a stereocenter (“What is the stereochemical result of an SN2 reaction?”). EPOCH never reveals the correct answer; students must attack the question repeatedly until they answer it correctly. EPOCH’s designers understand that students learn far more from struggling to answer a question correctly than from being told the correct answer. By contrast, most Web-based homework programs do not provide feedback for wrong answers, and most allow the student to give up before answering the question correctly (1, 3, 4). To our knowledge, the only program that provides feedback to students that is based on their free responses (not multiple-choice responses) is the Quantum Tutor (2). Overview of EPOCH’s Program Architecture EPOCH is a Java-based Web application. It consists of four distinct custom-built modules: the student interface (SI), the instructor tool (IT), the authoring tool (AT), and the back end. The commercial products MarvinSketch (7) (a structure-drawing tool), JChem (8) (a cheminformatics library), and Oracle (9) (a database program) are also integral parts of EPOCH. EPOCH runs on an Apache Tomcat Web server under the Linux operating system. Users access EPOCH over 164
Journal of Chemical Education
•
the Web with browsers running on their local personal computer with Windows, Macintosh, or Linux operating systems.
The Student Interface SI is the tool with which students answer questions. A student enters SI by choosing an assignment that the instructor has assembled (see the instructor tool, below). Each question in the assignment is color-coded according to whether the student has answered it correctly already, has worked on it but not answered it correctly, or has not yet worked on it (Figure 1). When a student chooses to work on a question, SI presents the question along with MarvinSketch (Figure 2). The student draws a structure in response and submits that. EPOCH then offers feedback to the student based on that response. If the response is wrong, the student can try again or can move to another question. EPOCH counts how many times the student submits a response to each question. A student can return to a question not yet answered correctly at any time until the due date specified by the instructor. Each time a student returns to a question, SI presents the previous response. Even after answering a question correctly, a student may return to the question later and work it again; in this case, SI does not show the correct response, but EPOCH remembers that the student has already answered the question correctly.
Figure 1. Screenshot of an assignment overview within the student interface.
Vol. 83 No. 1 January 2006
•
www.JCE.DivCHED.org
Information
The Instructor Tool IT is the tool that the instructor uses to assemble assignments (Figure 3). An instructor may take any questions from the EPOCH database of questions and assemble them as an assignment in any order. The instructor chooses the maximum number of times that EPOCH should allow a student to try to answer each question in the assignment. The instructor may make a fixed assignment, in which every student sees the same questions, or a random assignment, in which each student sees some number of questions chosen from a set that the instructor specifies. The database currently contains about 400 questions, and it continues to grow. The questions in the database are mostly taken from the end-of-chapter questions of the textbooks by Bruice and Wade (10); IT permits an instructor to construct assignments that are compatible with any textbook. IT also maintains a gradebook for the class (not shown). The instructor may view the score of each student on each assignment. If an instructor chooses to look more closely at any single assignment, IT shows each student’s score on each question in that assignment, provides a hyperlink to each student’s last response to each question, and offers statistics on how well the students are performing on each question. The hyperlink to the student’s last response is especially useful when the instructor answers e-mail from a student frustrated by a particular question.
•
Textbooks
•
Media
•
Resources
The Authoring Tool AT, the most complex component of EPOCH, is the tool that an instructor or other knowledgeable person uses to add questions to the database or modify existing database questions. Unless an instructor is deemed a “master author”, EPOCH makes an instructor’s changes to the database visible only within that instructor’s courses. The question database is organized into chapters; each chapter contains one or more problem sets, and each problem set contains one or more questions (Figure 4). Each question consists of a statement, a figure (a molecular structure, a half-reaction, or another image), and a series of conditions that EPOCH uses to analyze each response (Figure 5). Each condition is associated with feedback that EPOCH provides to a student whose response satisfies that condition. EPOCH
Figure 4. Screenshot of a problem set overview window within the authoring tool module. Figure 2. Screenshot of a question-solving window within the student interface module.
Figure 3. Screenshot of an assignment-assembly window within the instructor tool module.
www.JCE.DivCHED.org
•
Figure 5. Screenshot of a question-editing window within the authoring tool module.
Vol. 83 No. 1 January 2006
•
Journal of Chemical Education
165
Information
•
Textbooks
•
Media
•
Resources
tests a response against each condition in turn, beginning with the conditions that determine whether the response is correct or partially correct. If a response satisfies no conditions, EPOCH reports that the answer is incorrect, but it offers no meaningful feedback. The authoring tool also enables an instructor or author to view all student responses to a question. Authors and instructors can use this information to redesign the conditions and feedback to better match actual responses. Our philosophy is that the question database should not be static, rather it should evolve to meet the needs of the users.
EPOCH behind the Scenes In EPOCH, students and authors draw chemical structures with the commercial program MarvinSketch (7). Many programs that allow users to draw chemical structures already existed when we began this project in 2002, and many methods for representing the chemical structures in a standard format also existed (11). Rather than repeating work that many others had already done, we decided to incorporate an existing program into our own. We chose MarvinSketch because we could deliver it as an applet over the Web for the user to manipulate within a browser (7). JChem, a cheminformatics library, works hand in hand with MarvinSketch (8). This “chemical intelligence” program calculates the properties of chemical structures (formula, molecular weight, number of rings) and determines whether two structures are the same or different. The information generated by EPOCH (questions, assignments, responses, grades) is stored in an Oracle database (9). The back end of EPOCH coordinates communications among SI, IT, AT, JChem, and Oracle. EPOCH’s modules are embedded within a coursemanagement package. Two options are available: an instructor may choose to use EPOCH embedded within the commercial package Blackboard (3) or within course-management software that we have devised ourselves. The two course-man-
agement packages use the same EPOCH components and a common question database. Both course-management packages distinguish between student and instructor accounts. The EPOCH administrator verifies the credentials of individuals requesting an instructor account in the course-management program. Both coursemanagement programs require that an instructor create a course, enroll students in the course, assemble assignments, and make the assignments visible to students before students have access to EPOCH questions. Unfortunately, this model does not allow for a highly motivated individual student to access EPOCH questions without the aid of an instructor. Methods for Analyzing Student Responses EPOCH’s two most important features are that it accepts graphical input and that it provides response-specific feedback. To our knowledge, no other homework program has both features. We have developed a wide variety of condition types with which EPOCH can evaluate student responses so as to provide feedback that is as germane as possible (List 1). Most condition types rely on JChem to evaluate the responses (8). JChem uses established computational methods to analyze and compare structures. Most of the condition types are selfexplanatory, but some require a bit more elaboration.
If the Answer Is Exactly … This condition tests if a response exactly matches a structure drawn by the author. If the author’s structure contains a stereocenter, and the author has specified its configuration as R, S, or a mixture of R and S, then the stereocenter in the response must have the same configuration. If, on the other hand, the author has not specified the configuration of the stereocenter, the stereocenter in the response may have any configuration. The case is similar for stereogenic π bonds (alkenes, imines).
List 1. Response Condition Types [and Their Negatives] Supported by EPOCH 1. If the answer is [not] exactly... 2. If the answer is [not] exactly (without normalization)... 3. If the answer is either [neither] enantiomer of... 4. If the answer contains [doesn’t contain] (among other molecules)... 5. If the answer contains [doesn’t contain] the substructure... 6. If the answer contains [doesn’t contain] the skeleton... 7. If the answer has [doesn’t have] the same σ-bond network as... 8. If the answer contains [doesn’t contain] the functional group... 9. If the answer has [doesn’t have] the molecular formula... 10. If the number of ⵧ atoms is {[not] equal to, [not] more than, [not] less than}... 11. If the molecular weight is [not]... 12. If the exact mass is [not]... 13. If the number of rings is {[not] equal to, [not] more than, [not] less than}... 14. If the charge is {[not] equal to, [not] more than, [not] less than}... 15. If the number of molecules is {[not] equal to, [not] more than, [not] less than}... Items in {braces} represent pop-up menu selection items. The box (ⵧ) represents a text input box.
166
Journal of Chemical Education
•
Vol. 83 No. 1 January 2006
•
www.JCE.DivCHED.org
Information
If the Answer Is Exactly (without Normalization) … Aromatic compounds and ylides are commonly written in more than one resonance form, so EPOCH usually converts aromatics and ylides in authors’ structures and students’ responses into a single, standard resonance form before asking JChem to compare them. If, however, the author is asking a student to draw a resonance structure of, say, Me2S+–O–, the author may use this condition to forego normalization so that JChem can distinguish responses of Me2S=O and Me2S+–O–.
If the Answer Contains the Substructure …
•
Textbooks
•
Media
•
Resources
• Cyclohexene has the skeleton of cyclohexane (but not vice versa), because the orders of all the C–C bonds in cyclohexene are equal to or higher than those in cyclohexane • Bicyclo[3.1.0]hexane does not have the skeleton of cyclohexane, because an extra bond between two skeletal C atoms is not permissible • Cyclohexene oxide has the skeleton of cyclohexanol, because an extra bond between a skeletal C atom and a skeletal heteroatom is permissible
A response contains a substructure drawn by the author
We understand that others may define the concept of “skeleton” differently, but we find that this definition suits our purposes.
• Every non-H atom that is present in the substructure is also present in the response
If the Answer Has the Same σ-Bond Network As …
• Every H atom that is explicitly drawn in the substructure is present in the response, either implicitly or explicitly
This condition is also designed to test for resonance structures. Two structures with the same σ-bond network and the same charge must be resonance structures of one another. A student’s response has the same σ-bond network as a structure drawn by the author if every atom and σ bond in the response is in the author’s structure, and vice versa.
if:
• Every bond in the substructure is present in the response, and it has the same order (single, double, triple, or aromatic)
If the Answer Contains the Functional Group …
If the Answer Contains the Skeleton … The concept of the skeleton is drawn from natural products chemistry, in which two compounds differing only in one or more OH or OAc groups or π bonds are considered to have the same skeleton. A student’s response contains a skeleton drawn by the author if: • Every non-H atom that is present in the skeleton is also present in the response • Every H atom that is explicitly drawn in the skeleton is present in the response, either implicitly or explicitly
The concept of the functional group is essential to organic chemistry. Many functional groups are defined not only by the presence but also by the absence of certain atoms or groups, so a substructure search is not adequate for finding them. Because JChem does not incorporate the concept of functional groups, we have written our own Prolog program to search for them. Our language for defining functional groups is simple and flexible, allowing us to add more definitions to EPOCH over time and to modify existing ones if necessary. The definition of alcohol is illustrative. C1–O1–H C1O or C1N
• Every bond in the skeleton is present in the response at the same or higher order • In the response, no nonskeletal C atom is bound to a skeletal C atom (but a nonskeletal C atom may be bound to a skeletal heteroatom, and a nonskeletal heteroatom may be bound to a skeletal C atom) • The response may not contain additional σ bonds between skeletal C atoms (but the response may contain additional σ bonds between skeletal C atoms and skeletal or nonskeletal heteroatoms)
Examples Regarding Skeletons Skeletons may or may not be present. For example: • Toluene does not have the skeleton of benzene, because the additional C atom of toluene is bound to a benzene C atom • Anisole has the skeleton of benzene, because anisole has only a heteroatom attached to a benzene C atom • Anisole has the skeleton of phenol, because the additional C atom of anisole is bound to a heteroatom, not a C atom, of phenol • Benzene does not have the skeleton of phenol, because benzene lacks the O atom
www.JCE.DivCHED.org
•
This definition states that an alcohol is a structure with a C–O–H group in which the C atom is not doubly bound to O or N. Our definition of alcohol excludes carboxylic acids and the N,O tautomers of amides, but it includes enols, phenols, and hemiacetals, among others. EPOCH provides a list of about 130 functional groups from which the author can choose. Because functional-group definitions have not been standardized, it is sometimes not clear whether EPOCH will find a particular functional group within any given structure. (Is an oxime also an imine?) We have put a functional-group finder in EPOCH’s online manual so that an author can determine what functional groups EPOCH finds in a particular structure. A condition type that handles responses containing atoms with incorrect valences (pentavalent C atoms and the like) is glaringly absent from our list. We have not designed a condition type to handle such errors because MarvinSketch already handles them well. MarvinSketch draws a red line under atoms that have incorrect valences, and it automatically rectifies the H count of an atom when the user draws new bonds to an atom or changes the charge on it. Furthermore, MarvinSketch allows the user to choose to show a structure with all of the H atoms omitted, with just the H atoms attached to internal
Vol. 83 No. 1 January 2006
•
Journal of Chemical Education
167
Information
•
Textbooks
•
Media
•
Resources
C atoms omitted (default), with C–H bonds omitted but the H atoms shown (in the form CH, CH2, or CH3), or with all H atoms and bonds to H shown. We concede that it is pedagogically more valuable to require students to rectify H atoms themselves, even if they make errors, than it is to allow MarvinSketch to do it for them. However, we hope that students who use MarvinSketch will learn proper valences and H atom rectification by practice and observation. Providing Relevant Feedback An author uses chemistry knowledge and experience with teaching both to choose the conditions that EPOCH uses to evaluate the students’ responses and to write feedback appropriate to the responses that satisfy those conditions. For example, consider the question, Me2C=CH2 + HCl → ? The correct answer is t-butyl chloride, and isobutyl chloride is an incorrect but fairly reasonable response. Students, however, are incredibly imaginative when it comes to producing incorrect responses: a response might retain a π bond, contain two Cl atoms, contain more or fewer than four C atoms, contain four linear C atoms, or be positively charged. A properly authored set of conditions, such as the ones shown in Figure 5 and Table 1, captures all of these possibilities. The feedback for each condition gently points out the error in the student’s reasoning without giving away the correct answer. The goal is for students both to learn from their mistakes and to determine the correct answer themselves. EPOCH tests a response against each condition in the order in which the author lists the conditions. For example, by the time EPOCH reaches incorrect condition 6 in Table 1, it has already determined that the response is a single, neutral molecule with one Cl atom, no alkene, and the skeleton
of isobutane. Condition 6 rules out the possibility that the student has incorporated extraneous atoms such as O into the response. After condition 6, the only possible response left is isobutyl chloride. Some of the conditions are redundant—the author could omit conditions 3 and 4, because any response that satisfies either one also satisfies condition 6—but redundant conditions allow EPOCH to provide feedback as relevant as possible to the student’s error. If a response satisfies no conditions, EPOCH provides default feedback: “Sorry, no feedback can be offered for your answer. Want to try again?” Such feedback is usually a sign either of a wildly inaccurate response or of poorly designed conditions. Student Reactions to Using EPOCH Any program that induces students to do their homework should improve their performance. A program that does not allow students to look up the correct answers, compelling them instead to arrive at the correct answers themselves should improve student performance even more. A program that provides guidance to students so that they can see why their incorrect answers are incorrect and how to solve a problem correctly should improve their performance even more. We used EPOCH in a first-semester organic chemistry class of 200 students in Fall 2003. We motivated the students to use EPOCH by making their average score on the EPOCH problems 10% of their overall grade if it improved their grade. We did not limit the number of tries required to arrive at the correct answer for each question, so it was easy for students to earn 100% on the EPOCH problems. We found a correlation, though not an extremely strong one, between the score on the EPOCH questions and the score on exams (Figure 6 is representative). The weakness of the correlation was prob-
Table 1. Typical Conditions in Using EPOCH for Analyzing Responses to the Question What Product Will This Reaction Yield? (Me2C=CH2 HCl →) Correct and Partially Correct Answer Conditions
Feedback for Correct and Partially Correct Answers
If the answer is exactly isobutyl chloride
Right! Markovnikov addition of HCl across the alkene occurs, so the Cl atom adds to the more substituted C atom of the alkene.
Incorrect Answer Conditions
Feedback for Incorrect Answers for a Given Condition
1.
If the charge on the answer is not equal to 0
Please draw a neutral product.
2.
If the number of molecules in the answer is not equal to 1
Please draw a single, organic product.
3.
If the number of Cl atoms is not equal to 1
Exactly one equivalent of HCl reacts with the starting alkene. How many Cl atoms should the product have?
4.
If the answer has the functional group alkene
The electrons in the π bond are used to make a σ bond, so the π bond is consumed in the reaction.
5.
If the answer doesn’t have the skeleton isobutane
A skeletal rearrangement is unlikely to occur. No C–C bonds are made or broken in this reaction.
6.
If the answer doesn’t have the formula C4H9Cl
The product contains only the atoms found in the starting materials.
7.
If the answer is exactly isobutyl chloride
Which C atom of the alkene is better able to bear a positive charge? So to which C atom should the Cl atom be attached in the product?
1.
168
Journal of Chemical Education
•
Vol. 83 No. 1 January 2006
•
www.JCE.DivCHED.org
Information
•
Textbooks
•
Media
•
Resources
ably due to the fact that exam questions included mechanisms, retrosynthesis, and explanations, none of which were question types within EPOCH’s scope. We are now working to expand EPOCH to include such questions. Student perceptions are another measure of program effectiveness. In a survey to which 179 students responded anonymously: • 66% of the students said that the EPOCH questions were much more useful than the questions in the book, and 29% said they were a little more useful • 56% said the EPOCH questions were very helpful for their performance on exams, and 42% said they were somewhat helpful • Even if their second-semester instructor would not give credit for doing EPOCH questions, 49% said they would definitely do them, and 41% said they would probably do them • If their next instructor would not offer EPOCH, 34% would clamor until it was offered, 53% would ask for it just once, 12% would transfer into a section where it was offered, and only 2% would breathe a sigh of relief • 55% of the students said they enjoyed using EPOCH a lot, 36% liked it some, 8% were neutral, and fewer than 1% disliked or hated using it
Commercialization of EPOCH
Acknowledgments We thank Prentice–Hall, Inc., for funding this work. We also thank ChemAxon, Inc., for its generous cooperation and responsiveness throughout this project. Note
Prentice–Hall has funded the development of EPOCH. Prentice–Hall has subsequently renamed the system ACE Organic, redesigned the look of the interface, and deployed ACE Organic on a Prentice–Hall computer system at http:// www.prenhall.com/aceorganic. Instructors should contact their Prentice–Hall representative (http://vig.prenhall.com/ replocator) to obtain an ACE account. Students who purchase a new Prentice–Hall text receive access to ACE at no charge; other students can purchase access to ACE for a reasonable fee. At the time of this writing, Prentice–Hall does not restrict access to ACE to users of their textbooks. (Instructors using other publishers’ textbooks may wish to author new EPOCH questions more closely related to those textbooks; we believe the content of all organic chemistry textbooks is sufficiently similar that instructors can use EPOCH “off the shelf ” with almost any textbook.) Conclusion We have developed a Web-based organic chemistry homework program, EPOCH, whose two most important features are that students draw structures as responses to posed questions, and then students receive feedback tailored specifically to their responses. We believe that EPOCH is a tool that can have a real effect on student learning. There is no question that one-on-one interaction with a professor or with knowledgeable peers is the best way for a student to learn organic chemistry. However, for students who want sources of help in addition to their instructor, for students who are disinclined to spend much time with their instructor, for students who must work at odd hours or prefer to do so, and for instructors who don’t have time to spend with every one of their hundreds of students, we believe that EPOCH is the next best thing.
www.JCE.DivCHED.org
Figure 6. Correlation between students’ EPOCH scores (maximum 42) and performance on an exam as demonstrated by exam score.
•
1. This statement was true at the time of submission of this paper in mid-2004.
Literature Cited 1. Online Web-Based Learning (OWL) Home Page. http:// ccbit.cs.umass.edu/owl (accessed Jul 2005). 2. Quantum Tutors, Quantum Simulations, Inc. Home Page. http:// www.quantumsimulations.com (accessed Jul 2005). 3. Blackboard, Blackboard Inc. Home Page. http:// www.blackboard.com (accessed Jul 2005). 4. WebCT, WebCT, Inc. Home Page. http://www.webct.com (accessed Jul 2005); CyberProf Home Page. http://www.howhy.com/ home (accessed Jul 2005); Diploma, Brownstone Research Group Home Page. http://www.brownstone.net (accessed Jul 2005). 5. Penn, J. H.; Nedeff, V. M.; Gozdzik, G. J. Chem. Educ. 2000, 77, 227. 6. John McMurry Demo Companion Site. http:// wadsworthmedia.com/chemistry/mcmurry_site/chemistry_d/ booksite.html (accessed Jul 2005). 7. MarvinSketch, ChemAxon Ltd. Home Page. http:// www.chemaxon.com/marvin (accessed Jul 2005). 8. JChem Base, ChemAxon Ltd. Home Page. http:// www.chemaxon.com/jchem/ (accessed Jul 2005). 9. Oracle, Oracle Corp. Home Page. http://www.oracle.com (accessed Jul 2005). 10. Bruice, P. Y. Organic Chemistry, 4th ed.; Prentice–Hall: Upper Saddle River, NJ, 2003. Wade, L. G. Organic Chemistry, 5th ed.; Prentice–Hall: Upper Saddle River, NJ, 2003. 11. CTFile Formats; October, 2003. MDL Information Systems, Inc. PDF download. http://www.mdl.com/downloads/public/ctfile/ ctfile.pdf (accessed Jul 2005).
Vol. 83 No. 1 January 2006
•
Journal of Chemical Education
169
