No CrossRef data available.
Article contents
Commensurability engineering is first and foremost a theoretical exercise
Published online by Cambridge University Press: 05 February 2024
Abstract
I provide a personal perspective on metastudies and emphasize lesser-known benefits. I stress the need for integrative theories to establish commensurability between experiments. I argue that mathematical social scientists should be engaged to develop integrative theories, and that likelihood functions provide a common mathematical framework across experiments. The development of quantitative theories promotes commensurability engineering on a larger scale.
- Type
- Open Peer Commentary
- Information
- Copyright
- Copyright © The Author(s), 2024. Published by Cambridge University Press
References
Baribault, B. (2019). Using hierarchical Bayesian models to test complex theories about the nature of latent cognitive processes (Doctoral thesis). University of California, Irvine. ProQuest ID: Baribault_uci_0030D_15781. Merritt ID: ark:/13030/m5cc62b5. Retrieved from https://escholarship.org/uc/item/31t8r1k3Google Scholar
Baribault, B., Donkin, C., Little, D., Trueblood, J., Oravecz, Z., van Ravenzwaaij, D., … Vandekerckhove, J. (2018). Meta-studies for robust tests of theory. Proceedings of the National Academy of Sciences of the United States of America, 115, 2607–2612.CrossRefGoogle Scholar
Cronbach, L. J., Rajaratnam, N., & Gleser, G. C. (1963). Theory of generalizability: A liberalization of reliability theory. British Journal of Statistical Psychology, 16(2), 137–163.CrossRefGoogle Scholar
DeKay, M. L., Rubinchik, N., Li, Z., & De Boeck, P. (2022). Accelerating psychological science with metastudies: A demonstration using the risky-choice framing effect. Perspectives on Psychological Science, 17(6), 1704–1736. https://doi.org/10.1177/17456916221079611CrossRefGoogle ScholarPubMed
Kenett, R. S., & Rubinstein, A. (2021). Generalizing research findings for enhanced reproducibility: An approach based on verbal alternative representations. Scientometrics, 126, 4137–4151. https://doi.org/10.1007/s11192-021-03914-1CrossRefGoogle Scholar
Rubinchik, N. (2019). A demonstration of the meta-studies methodology using the risky-choice framing effect (Master's thesis). Ohio State University. OhioLINK Electronic Theses and Dissertations Center. Retrieved from http://rave.ohiolink.edu/etdc/view?acc_num=osu1574201911927335Google Scholar
Simons, D. J., Shoda, Y., & Lindsay, D. S. (2017). Constraints on generality (COG): A proposed addition to all empirical papers. Perspectives on Psychological Science, 12(6), 1123–1128.CrossRefGoogle ScholarPubMed
Watts, D. (2017). Should social science be more solution-oriented? Nature Human Behavior, 1, 15. https://doi.org/10.1038/s41562-016-0015CrossRefGoogle Scholar
Target article
Beyond playing 20 questions with nature: Integrative experiment design in the social and behavioral sciences
Related commentaries (31)
Against naïve induction from experimental data
Are language–cognition interactions bigger than a breadbox? Integrative modeling and design space thinking temper simplistic questions about causally dense phenomena
Assume a can opener
Beyond integrative experiment design: Systematic experimentation guided by causal discovery AI
Commensurability engineering is first and foremost a theoretical exercise
Confidence in research findings depends on theory
Consensus meetings will outperform integrative experiments
Dimensional versus conceptual incommensurability in the social and behavioral sciences
Discovering the unknown unknowns of research cartography with high-throughput natural description
Diversity of contributions is not efficient but is essential for science
Don't let perfect be the enemy of better: In defense of unparameterized megastudies
Eliminativist induction cannot be a solution to psychology's crisis
Experiment commensurability does not necessitate research consolidation
Explore your experimental designs and theories before you exploit them!
Getting lost in an infinite design space is no solution
Individual differences do matter
Integrative design for thought-experiments
Integrative experiments require a shared theoretical and methodological basis
Is generalization decay a fundamental law of psychology?
Measurement validity and the integrative approach
Neuroadaptive Bayesian optimisation can allow integrative design spaces at the individual level in the social and behavioural sciences… and beyond
Phenomena complexity, disciplinary consensus, and experimental versus correlational research in psychological science
Representative design: A realistic alternative to (systematic) integrative design
Sampling complex social and behavioral phenomena
Some problems with zooming out as scientific reform
Test many theories in many ways
The elephant's other legs: What some sciences actually do
The future of experimental design: Integrative, but is the sample diverse enough?
The miss of the framework
The social sciences needs more than integrative experimental designs: We need better theories
There are no shortcuts to theory
Author response
Replies to commentaries on beyond playing 20 questions with nature