The foolish astronomer

An astronomer is interested in building a telescope to study a distant galaxy. A critical factor in the design of the telescope is its magnification power. Seen through a telescope with insufficient power, the galaxy will appear as an indecipherable blur. But rather than figure out how much power he needs to make his observations, the astronomer foolishly decides to build a telescope on the basis of available funds. Maybe he does not know how much magnification power he needs, but he knows exactly how much money is in his equipment budget. So he orders the biggest telescope he can afford and hopes for the best.

In social science research the foolish astronomer is the one who sets sample sizes on the basis of resource availability. He is the one who, when asked “how big should your sample be?”, answers “as big as I can afford.” Resource constraints are a fact of research life. But if our goal is to conserve limited resources, it is essential that we begin our studies by asking questions about their power to detect the phenomena we are seeking. How big a sample size do I need to test my hypotheses? Assuming the phenomenon I'm searching for is real, what are my chances of finding it given my research design? How can I increase my chances?