Chapter overview

As stated in Chapter 1, designing content “begins with the end in mind.” The goal of medical education is to obtain maximum levels of learning. Chapter 4 discusses how to measure whether that endpoint has been met. Program evaluation is a hot topic in medical education these days, and this chapter analyzes not only the methods for measuring educational outcomes but also the results of applying these methods. The goal is to determine at a minimum whether learning has occurred, and, ideally, whether behavior has changed in response to an educational program.

The first section discusses the principles of evaluating educational programs, including methods for determining whether the program has been successful. Five levels of evaluation are proposed – audience reaction, learning, behavioral change, results, and return on education – as the current academic standards for any training program. These specific outcomes are in fact the endpoints that should be targeted when the program is first designed.

The next section explains that, for a medical education program, Level 1 success (audience reaction) is the usual standard of evaluation and is quite rudimentary. Level 1 evaluation merely determines whether audiences liked the learning event, and thus is little more than a “smile sheet.” Other Level 1 measurements can include simply counting the number of people who attended or asking learners whether they thought the education objectives were met.

The third section discusses how to document whether Level 2 success has occurred: namely, learning.