Evaluation in Instructional Design

This summer, my district will be rewriting our 9th grade science curriculum in an effort to deeply integrate engineering into our courses.  The ultimate goal is to increase the number of our graduates who go on to careers in STEM, but the success of our reforms will largely be tracked by student performance on the nature of engineering strands of the state science test.  The trick is students take the test in the spring of their 10th grade year, so we won't receive our first round of results until August 2016, nearly two years after we first begin implementation.  We will need to find other ways to evaluate the success of our new instructional methods in order to refine our instruction in a much more timely manner.  The formative evaluation phase of instructional design can provide an effective framework for us to determine what is (and isn't) working.

Variations on several phases of formative evaluation are already in place, so our task will be to ensure we are taking full advantage of these opportunities.  Several of us will spend a portion of the summer with STEM education experts at the University of Minnesota to begin developing materials and lessons.  As part of this program, we will be presenting the developed lessons to students in a STEM summer camp.  By soliciting reflection from the summer camp instructors and requesting feedback from the student participants, the STEM camp can become an effective opportunity for small group evaluation and provide a basis for the first round of revisions.

At the end of the summer, the lessons and materials developed at the University of Minnesota will be shared with the remaining 9th grade science teachers at a series of curriculum writing meetings prior to the start of the school year.  The teachers who do not help develop the materials can be leveraged as subject matter experts.  They will provide a fresh set of eyes to evaluate the new curriculum and provide input on further revisions.

While these steps should put us in a good position for successful implementation of the new curriculum in the fall, that does not mean we should consider our evaluation complete.  The first year can be approached as a field trial.  During our summer curriculum writing, we should develop student attitude surveys, especially since one of the desired outcomes is more students pursuing STEM, and assessment tools based on sample items from the state science test.  We can also develop surveys and reflection tools for ourselves and the other teachers involved.  Since each 9th grade science course is offered every trimester for a total of three times per year, a thoughtful, intentional evaluation process will allow us to continually revise and evaluate lessons and materials.

By incorporating evaluation strategies that are standard to the instructional design process, we can take steps to painlessly and effectively identify what is working and what isn't in our reforms.  Then, we can focus our efforts to improve the weakest aspects of our lessons and materials to ensure our reforms can meet our desired outcomes.