Overview

If the audience and goals for your evaluation have led you to choose a naturalistic design, you will need to plan on spending a significant amount of time gathering the evaluation data. These data can be recorded in a variety of ways and can describe the program from a variety of points of view. The most common methods for gathering and recording data in a naturalistic program evaluation are observation, interviews, journals, questionnaires, and document analysis. The data that are recorded by these methods can come from a variety of sources: students, instructors, administrators, evaluators, and other persons who interact with the program. Once the data have been collected, you will need to approach the analysis and interpretation of these data with the same systematicity and thoroughness required to gather them. Unlike quantitative analysis, there are no set procedures, defined in detail, for the evaluator to follow. There are, however, basic principles and guidelines that inform the analysis of qualitative data for the purposes of program evaluation. The procedures that I suggest in this chapter consist of the following stages: developing a thematic framework, organizing the data, coding the data, reducing the data, and interpreting the data. These analytic procedures are not a linear series of steps, but are accomplished in an iterative fashion – for example, they include returning to the coding stage after reducing the data.