Chapter 11
Drawing Meaningful Conclusions from Education Experiments Melanie M. Cooper Downloaded by UNIV OF GUELPH LIBRARY on July 3, 2012 | http://pubs.acs.org Publication Date: January 3, 2008 | doi: 10.1021/bk-2008-0976.ch011
Department of Chemistry, Clemson University, Clemson, SC 29634
The ultimate goal of a chemistry education research project is that the scholarly work resulting from the research will have an impact on our understanding of teaching and learning, and will result in more effective and meaningful learning in the chemistry classroom. To do this the researcher must draw meaningful conclusions from the data, situate them within the context of previous work, and discuss their implications. It is particularly important that the researcher be aware of the kinds of errors that can creep into educational research, and this chapter describes a number of common problems often found in research reports in chemistry education. These include mistaking cause and effect, overgeneralization, using anecdotal evidence, not controlling for differences in student population, mistaking self-reported learning for actual improvements in student learning, and disturbing the test population by the act of investigation. Examples of exemplary reports using qualitative, quantitative, and mixed method research designs are provided to show how these problems can be avoided.
© 2008 American Chemical Society
In Nuts and Bolts of Chemical Education Research; Bunce, D., et al.; ACS Symposium Series; American Chemical Society: Washington, DC, 2008.
171
Downloaded by UNIV OF GUELPH LIBRARY on July 3, 2012 | http://pubs.acs.org Publication Date: January 3, 2008 | doi: 10.1021/bk-2008-0976.ch011
172 The preceding chapters in this book have discussed the major elements required for exemplary chemistry education research. Examples include: Zare (/) asked important questions in Chapter 2 that can be answered by research. Bunce (2) explained how to ask good questions in Chapter 4, which leads to designing appropriate experiments to approach the research questions, and using established techniques to analyze the data. As explained by Abraham and Williamson in Chapters 5 and 6 (3,4), the design of any experiment must be guided by theory. That is, there must be an underlying theory-base to any research, and any "treatment" administered must be based on that theory. Experimental designs may range from purely quantitative, as described by Sanger (5) in Chapter 8, where the data can be analyzed to produce statistically valid answers about what students learn, to qualitative designs (described by Bretz (6) in Chapter 7), where researchers directly study student behavior to try to understand why and how students learn. Mixed methods experiments that include both qualitative and quantitative techniques, described by Towns (7) in Chapter 9, can provide insights into both of these questions. Even i f a research study has been well designed - with a question that can be answered by the methods chosen, and the data then analyzed correctly, there still remain a number of pitfalls for the researcher in moving to the next step of making the work useful to others by drawing meaningful conclusions. This chapter focuses on how difficult it can be to drawing meaningful results from data. For example, researchers sometimes omit the last part of the scholarly process. They fail to ensure that the findings are explained and placed within the context of what is already known, and they fail to provide guidance and future directions. It has been said that "Research is a process for obtaining information, and scholarship is a process for converting information into knowledge."(
