Book contents
- Frontmatter
- Dedication
- Contents
- Preface and Acknowledgments
- Introduction
- Section I Thinking Like a Data Scientist
- Section II Communicating Like a Data Scientist
- Section III Applying the Tools of Data Science to Education
- 12 Waiting for Achilles
- 13 How Much Is Tenure Worth?
- 14 Detecting Cheating Badly: If It Could Have Been, It Must Have Been
- 15 When Nothing Is Not Zero: A True Saga of Missing Data, Adequate Yearly Progress, and a Memphis Charter School
- 16 Musing about Changes in the SAT: Is the College Board Getting Rid of the Bulldog?
- 17 For Want of a Nail: Why Worthless Subscores May Be Seriously Impeding the Progress of Western Civilization
- Section IV Conclusion: Don't Try Th is at Home
- Bibliography
- Sources
- Index
15 - When Nothing Is Not Zero: A True Saga of Missing Data, Adequate Yearly Progress, and a Memphis Charter School
from Section III - Applying the Tools of Data Science to Education
Published online by Cambridge University Press: 05 December 2015
- Frontmatter
- Dedication
- Contents
- Preface and Acknowledgments
- Introduction
- Section I Thinking Like a Data Scientist
- Section II Communicating Like a Data Scientist
- Section III Applying the Tools of Data Science to Education
- 12 Waiting for Achilles
- 13 How Much Is Tenure Worth?
- 14 Detecting Cheating Badly: If It Could Have Been, It Must Have Been
- 15 When Nothing Is Not Zero: A True Saga of Missing Data, Adequate Yearly Progress, and a Memphis Charter School
- 16 Musing about Changes in the SAT: Is the College Board Getting Rid of the Bulldog?
- 17 For Want of a Nail: Why Worthless Subscores May Be Seriously Impeding the Progress of Western Civilization
- Section IV Conclusion: Don't Try Th is at Home
- Bibliography
- Sources
- Index
Summary
One of the most vexing problems in all of school evaluation is missing data. This challenge manifests itself in many ways, but when a school, a teacher, or a student is being evaluated on the basis of their performance, the most common missing data are test scores.
If we want to measure growth we need both a pre- and a postscore. What are we to do when one, the other, or both are missing? If we want to measure school performance, what do we do when some of the student test scores are missing?
This problem has no single answer. Sometimes the character of the missingness can be ignored, but usually this approach is reasonable only when the proportion of missing data is very small. Otherwise the most common strategy is “data imputation” (discussed in Chapters 3 and 4, and its misuse was shown in Chapter 7). Data imputation involves deriving some plausible numbers and inserting them in the holes. How we choose those numbers depends on the situation and on what ancillary information is available.
Let us consider the specific situation of Promise Academy, an inner-city charter school in Memphis, Tennessee, that enrolled students in kindergarten through fourth grade for the 2010–11 school year. Its performance has been evaluated on many criteria, but of relevance here is the performance of Promise students on the state's reading/language arts (RLA) test. This score is dubbed its Reading/Language Arts Adequate Yearly Progress (AYP) and it depends on two components: the scores of third- and fourth-grade students on the RLA portion of the test and the performance of fifth-grade students on a separate writing assessment.
We observe the RLA scores, but because Promise Academy does not have any fifth-grade students, all of the writing scores are missing. What scores should we impute to allow us to calculate a plausible total score? The state of Tennessee's rules require a score of zero be inserted. In some circumstances imputing scores of zero might be reasonable. For example, if a school only tests half of its students, we might reasonably infer that it was trying to game the system by choosing to omit the scores from what are likely to be the lowest-performing students.
- Type
- Chapter
- Information
- Truth or TruthinessDistinguishing Fact from Fiction by Learning to Think Like a Data Scientist, pp. 161 - 166Publisher: Cambridge University PressPrint publication year: 2015