Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
  1. Home
  2. Books
  3. Images as Data for Social Science Research
  • Cited by 40
Publisher:
Cambridge University Press
Online publication date:
July 2020
Print publication year:
2020
Online ISBN:
9781108860741
DOI:
https://doi.org/10.1017/9781108860741
Subjects:
Research Methods in Politics, Politics and International Relations, Research Methods In Politics
Series:
Elements in Quantitative and Computational Methods for the Social Sciences
Information

Book description

Images play a crucial role in shaping and reflecting political life. Digitization has vastly increased the presence of such images in daily life, creating valuable new research opportunities for social scientists. We show how recent innovations in computer vision methods can substantially lower the costs of using images as data. We introduce readers to the deep learning algorithms commonly used for object recognition, facial recognition, and visual sentiment analysis. We then provide guidance and specific instructions for scholars interested in using these methods in their own research.

References

References

A Neural Network Playground (2020). https://playground.tensorflow.org/#activation=tanh&batchSize=10&dataset=circle&regDataset=reg-plane&lear ningRate=0.03&regularizationRate=0&noise=0&networkShape=4,2&se ed=0.81811&showTestData=false&discretize=false&percTrainData=50 &x=true&y=true&xTimesY=fal (last accessed April 26, 2020).
ACMFAccT-Links (2020). https://facctconference.org/links.html (last accessed April 26, 2020).
Anastasopoulos, L. J. et al. (2016). “Photographic Home Styles in Congress: A Computer Vision Approach,” Approach,” 1–52. http://arxiv.org/abs/1611.09942.
Bennett, W. L., and Segerberg, A (2013). The Logic of Connective Action: Digital Media and the Personalization of Contentious Politics. New York: Cambridge University Press.
Benoit, K., et al. (2016). “Crowd-sourced Text Analysis: Reproducible and Agile Production of Political Data.” American Political Science Review 110(2), 278–295.
Bimber, B., Flanagin, A. J., and Stohl, C (2005). “Reconceptualizing Collective Action in the Contemporary Media Environment.” Communication Theory 15(4), 365–388. http://doi.wiley.com/10.1111/j.1468–2885.2005.tb00340.x.
Brantner, C., Lobinger, K, and Irmgard, W (2011). “Effects of Visual Framing on Emotional Responses and Evaluations of News Stories about the Gaza Conflict 2009.” Journalism & Mass Communication Quarterly 88(3), 523–540.
Britz, D. (2015). Understanding Convolutional Neural Networks for NLP – WildML. www.wildml.com/2015/11/understanding-convolutional-neural-networks-for-nlp/ (last accessed April 26, 2020).
Broussard, M. (2018). Artificial Unintelligence: How Computers Misunderstand the World. Cambridge, MA: The Massachusetts Institute of Technology Press.
Budhiraja, A. (2016). Dropout in (Deep) Machine Learning-Amar Budhiraja – Medium. URL: https://medium.com/@amarbudhiraja/https-medium-com-amarbudhiraja-learning-less-to-learn-better-dropout-in-deep-mach ine-learning-74334da4bfc5 (last accessed April 26, 2020).
Buduma, N., and Locascio, N (2017). Fundamentals of Deep Learning: Designing Next-generation Machine Intelligence Algorithms. Sebastopol, CA: O’Reilly Media.
Buolamwini, J., and Gebru, T (2018). “Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification.” In: Proceedings of Machine Learning Research. Vol. 81, pp. 1–15.
Burns, N., et al. (2011). “Sentiment Analysis of Customer Reviews: Balanced versus Unbalanced Datasets.” In: Knowledge-Based and Intelligent Information and Engineering Systems Ed. by Konig, A et al. Berlin, Heidelberg: Springer Berlin Heidelberg, pp. 161–170.
Callen, M., and Long, J. D. (2015). “Institutional Corruption and Election Fraud: Evidence from a Field Experiment in Afghanistan.” American Economic Review 105(1), 354–381.
Cantú, F. (2019). “The Fingerprints of Fraud: Evidence from Mexico’s 1988 Presidential Election.” American Political Science Review. Vol. 113, Issue 3, pp. 710–726.
Casas, A., and Webb Williams, N (2018). “Images that Matter: Online Protests and the Mobilizing Role of Pictures.” Political Research Quarterly, Vol. 72, Issue 2, 360–375.
Casas, A., et al. (2019). “Visual Clustering: A Technique for Drastically Reducing Image Annotation Tasks.” Presented at Annual Meeting of the International Communication Association, Washington, D.C.
Castells, M. (2012). Networks of Outrage and Hope: Social Movements in the Internet Age. Cambridge, UK; Malden, MA: Polity Press.
Cillizza, C. (2018). Melania Trump’s ’I Really Don’t Care. Do U?’ Jacket Was No Mistake- – CNNPolitics. www.cnn.com/2018/06/21/politics/melania-trump-jacket-border-visit/index.html (last accessed April 26, 2020).
Clarke, K., and Kocak, K (2018). Replication Data for: Launching Revolution: Social Media and the Egyptian Uprising’s First Movers. https://doi.org/10.7910/DVN/S17DDJ.
COCO – Common Objects in Context (2020). http://cocodataset.org/#home (last accessed April 26, 2020).
Corrigall-Brown, C., and Wilkes, R (2012). “Picturing Protest: The Visual Framing of Collective Action by First Nations in Canada.” American Behavioral Scientist 56(2), 223–243.
CS231n Convolutional Neural Networks for Visual Recognition (2020). https://cs231n.github.io/ (last accessed April 25, 2020).
Dahmen, N. S. (2012). “Photographic Framing in the Stem Cell Debate.” American Behavioral Scientist 56(2), 189–203.
Dietrich, B.J. (2019). “Using Motion Detection to Measure Social Polarization in the U.S. House of Representatives.”
Domhan, T., Springenberg, J, and Hutter, F (2015). “Speeding Up Automatic Hyperparameter Optimization of Deep Neural Networks by Extrapolation of Learning Curves”. IJCAI’15: Proceedings of the 24th International Conference on Artificial Intelligence, pp. 3460–3468
Commission, European (2020). EU Data Protection Rules. https://ec.europa.eu/ info/priorities/justice-and-fundamental-rights/data-protection/2018-reform-eu-data-protection-rules/eu-data-protection-rules_en (last accessed April 26, 2020).
Geitgey, A. (2020). ageitgey/face_recognition: The World’s Simplest Facial Recognition API for Python and the Command Line. https://github.com/ageitgey/face_recognition (last accessed April 26, 2020).
Gelman, A., and Hill, J (2007). Data Analysis Using Regression and Multilevel/Hierarchical Models. New York, NY: Cambridge University Press.
Gender Shades (2020). http://gendershades.org/ (last accessed April 26,2020).
Girshick, R. B. (2015). “Fast {R-CNN}.” CoRR abs/1504.0. http://arxiv.org/abs/1504.08083.
Girshick, R. B., et al. (2013). “Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation.” CoRR abs/1311.2. http://arxiv.org/abs/1311.2524.
Gitlin, T. (1980). The Whole World Is Watching: Mass Media in the Making and Unmaking of the New Left. Berkeley: University of California Press.
Goh, G. (2017). “Why Momentum Really Works.” Distill. http://distill.pub/2017/momentum.
Golle, P. (2008). “Machine Learning Attacks against the Asirra CAPTCHA.” In: CCS ’08 Proceedings of the 15th ACM Conference on Computer and Communications Security, pp. 535–542.
Grabe, M. E., and Bucy, E. P. (2009). Image Bite Politics: News and the Visual Framing of Elections. Oxford; New York: Oxford University Press.
Guo, Y., et al. (2016). “MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition.” In: European Conference on Computer Vision (ECCV). www.hdwallpapers.in/anne.
Hamner, B. (2019). 2016 US Election | Kaggle. www.kaggle.com/benhamner/2016-us-election (last accessed April 26, 2020).
He, K., Gkioxari, G, et al. (2017). “Mask {R-CNN}.” CoRR abs/1703.0. http://arxiv.org/abs/1703.06870.
He, K., Zhang, X, et al. (2015). “Deep Residual Learning for Image Recognition.” In: arXiv:1512.03385.
Henderson, J. V., Storeygard, A, and Weil, D. N. (2012). “Measuring Economic Growth from Outer Space Author.” American Economic Review 102(2), 994–1028.
Horiuchi, Y., Komatsu, T, and Nakaya, F (2012). “Should Candidates Smile to Win Elections? An Application of Automated Face Recognition Technology.” Political Psychology 33(6), 925–933.
Howard, P. N., andHussain, M. M. (2013). Democracy’s Fourth Wave?: Digital Media and the Arab Spring. New York, NY: Oxford University Press.
Hwang, J., Imai, K, and Tarr, A (2019). “Automated Coding of Political Campaign Advertisement Videos: An Empirical Validation Study.” Kosuke Imai’s Homepage. https://imai.fas.harvard.edu/research/files/campvideo.pdf.
ImageNet (2020). http://image-net.org/ (last accessed April 26, 2020).
Internet Live Stats – Internet Usage & Social Media Statistics (2020). www.internetlivestats.com/ (last accessed April 26, 2020).
Introna, L. D., and Wood, D (2004). “Picturing Algorithmic Surveillance: The Politics of Facial Recognition Systems.” Surveillance and Society 2(2–3), 177–198.
Iyer, A., and Oldmeadow, J (2006). “Picture This: Emotional and Political Responses to Photographs of the Kenneth Bigley Kidnapping.” European Journal of Social Psychology 36(5), 635–647.
Jean, N., et al. (2016). “Combining Satellite Imagery and Machine Learning to Predict Poverty.” Science 353(6301), 790–794. www.ncbi.nlm.nih.gov/pubmed/27540167.
Joo, J., Bucy, E. P., and Seidel, C (2019). “Automated Coding of Televised Leader Displays: Detecting Nonverbal Political Behavior with Computer Vision and Deep Learning.” International Journal of Communication. Vol. 19, pp. 4044–4066.
Joo, J., Li, W, et al. (2014). “Visual Persuasion: Inferring Communicative Intents of Images.” In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp. 216–223.
Joo, J., Steen, F. F., and Zhu, S.-C. (2015). “Automated Facial Trait Judgment and Election Outcome Prediction: Social Dimensions of Face.” In: Proceedings of the IEEE International Conference on Computer Vision. IEEE, pp. 3712–3720.
Kaufman, A., King, G, and Komisarchik, M (2019). “How to Measure Legislative District Compactness If You Only Know It When You See it.”
Kearns, M., and Roth, A (2019). The Ethical Algorithm: The Science of Socially Aware Algorithm Design Oxford: Oxford University Press.
Kharroub, T., and Bas, O (2015). “Social Media and Protests: An Examination of Twitter Images of the 2011 Egyptian Revolution.” New Media & Society 18(9): 1973 –1992. http://nms.sagepub.com/cgi/doi/10.1177/1461444815571914.
Krizhevsky, A., Sutskever, I, and Hinton, G. E. (2012). “ImageNet Classification with Deep Convolutional Neural Networks.” In: Advances in Neural Information Processing Systems, pp. 1106–1114.
Kulkarni, G., et al. (2013). “BabyTalk: Understanding and Generating Simple Image Descriptions.” IEEE Transactions on Pattern Analysis and Machine Intelligence 35(12), 2891–2903.
Lam, O.,etal. (2019). Men Appear Twice as Often as Women in News Photos on Facebook. Tech. rep. Pew Research Center. www.journalism.org/2019/05/23/men-appear-twice-as-often-as-women-in-news-photos-on-facebook/.
University, Lancaster (2020). GDPR: What Researchers Need to Know | Lancaster University. www.lancaster.ac.uk/research/research-services/research-integrity-ethics-governance/data-protection/gdpr-what-researchers-need-to-know/ (last accessed April 26, 2020).
LeCun, Y., Bengio, Y, and Hinton, G (2015). “Deep Learning.” Nature 521(7553), 436–444.
Li, H., et al. (2015). “A Convolutional Neural Network Cascade for Face Detection.” In: 2015 IEEE Conference on Computer Vision and Pattern Recognition.
Lin, T. Y., et al. (2014). “Microsoft COCO: Common Objects in Context.” In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 8693 LNCS. PART 5. Springer Verlag, pp. 740–755.
Marcus, G. E., Neuman, W. R., and MacKuen, M (2000). Affective Intelligence and Political Judgement. Chicago and London: University of Chicago Press.
Mebane, W. R. J., et al. (2017). “Using Twitter to Observe Election Incidents in the United States.” (Paper presented at the 2016 Annual Meeting of the Midwest Political Science Association, Chicago, April 6–9, 2017)
Messaris, P., and Abraham, L (2001). “The Role of Images in Framing News Stories.” In: Framing Public Life: Perspectives on Media and Our Understanding of the Social World. Ed. by Reese, Stephen D., Gandy, Oscar H., and Grant, A. E.. Mahwah, NJ: Lawrence Erlbaum Associates Publishers, pp. 215–226.
Metz, C. (2019). Facial Recognition Tech Is Growing Stronger, Thanks to Your Face. www.nytimes.com/2019/07/13/technology/databases-faces-facial-recognition-technology.html?searchResultPosition=7.
Moreno, M. A., et al. (2013). “Ethics of Social Media Research: Common Concerns and Practical Considerations”. Cyberpsychology, Behavior, and Social Networking 16(9), 708–713. URL: http://online.liebertpub.com/doi/abs/10.1089/cyber.2012.0334.
Mountassir, A., Hourda, Benbrahim, and Berrada, I (2012). “An Empirical Study to Address the Problem of Unbalanced Data Sets in Sentiment Classification.” In: 2012 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 3298–3303.
MS-Celeb-1M: Challenge of Recognizing One Million Celebrities in the Real World-Microsoft Research (2020). www.microsoft.com/en-us/research/project/ms-celeb-1m-challenge-recognizing-one-million-celebrities-real-world/ (last accessed April 26, 2020).
Nanne, A., et al. (2019). “The Use of Computer Vision to Analyze Visual Brand-Related User Generated Content: A Comparison of YOLOV2, Google Cloud Vision, and Clarifai.” Journal of Interactive Marketing 50, 156–167.
Nelson, D. L., Reed, V. S., and Walling, J. R. (1976). “Pictorial Superiority Effect.” Journal of Experimental Psychology: Human Learning and Memory 2(5), 523–528.
O’Neil, C. (2017). Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy New York: Broadway Books.
Owen, D. (2018). “Should We Be Worried About Computerized Facial Recognition?” The New Yorker. www.newyorker.com/magazine/2018/12/17/should-we-be-worried-about-computerized-facial-recognition.
Paivio, A., Rogers, T. B., and Smythe, P. C. (1968). “Why Are Pictures Easier to Recall Than Words?” Psychonomic Science 11(4), 137–138.
Peng, K.-c., et al. (2015). “A Mixed Bag of Emotions: Model, Predict, and Transfer Emotion Distributions.” In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 1–9.
Peng, Y. (2018). “Same Candidates, Different Faces: Uncovering Media Bias in Visual Portrayals of Presidential Candidates with Computer Vision.” Journal of Communication 68(5), 920–941.
Peng, Y. (2020). COMPUTER VISION – YILANG PENG, https://yilangpeng.com/ computer-vision/ (last accessed April 26, 2020).
Philipp, H., Müller-Crepon, C, and Cederman, L.-E. (n.d.). “Roads to Rule, Roads to Rebel: Relational State Capacity and Conflict in Africa.”
Powell, T., et al. (2015). “A Clearer Picture: The Contribution of Visuals and Text to Framing Effects.” Journal of Communication 65(6), 997–1017.
Project Jupyter | Home (2020). https://jupyter.org/ (last accessed April 26, 2020).
Raiford, L. (2007). “World Together: SNCC and Photography of the Civil Rights Movement.” American Quarterly 59(4), 1129–1157.
Redmon, J., Divvala, S, et al. (2016). “You Only Look Once: Unified, Real Time Object Detection.” In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 779–788.
Redmon, J., and Farhadi, A (2018). “YOLOv3: An Incremental Improvement.” https://arxiv.org/abs/1804.02767.
Ren, S., et al. (2015). “Faster {R-CNN:} Towards Real-Time Object Detection with Region Proposal Networks.” CoRR abs/1506.0. http://arxiv.org/abs/1506.01497.
Ribeiro, M. T., Singh, S, and Guestrin, C (2016). “ ‘Why Should I Trust You?’: Explaining the Predictions of Any Classifier.” In: Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. KDD ’16. New York, N: ACM, pp. 1135–1144. http://doi.acm.org/10.1145/2939672.2939778.
Rosenberg, M., Confessore, N, and Cadwalladr, C (2018). “How Trump Consultants Exploited the Facebook Data of Millions.” The New York Times. www.nytimes.com/2018/03/17/us/politics/cambridge-analytica-trump-campaign.html.
Rosenberg, S. W., et al. (1986). “The Image and the Vote: The Effect of Candidate Presentation on Voter Preference.” American Journal of Political Science 30(1), 108–127.
Ruder, S. (2016). “An Overview of Gradient Descent Optimization Algorithms.” CoRR abs/1609.0. http://arxiv.org/abs/1609.04747.
Russakovsky, O., et al. (2015). “ImageNet Large Scale Visual Recognition Challenge.” International Journal of Computer Vision (IJCV) 115(3), 211–252.
Saldaña, J. (2009). The Coding Manual for Qualitative Researchers. Thousand Oaks, CA: Sage Publications Ltd.
Schmidhuber, J. (2015). “Deep Learning in Neural Networks: An Overview”. Neural Networks 61, 85–117.
Simonite, T. (2017). Machines Learn a Biased View of Women. www.wired .com/story/machines-taught-by-photos-learn-a-sexist-view-of-women/ (last accessed April 26, 2020).
Simonite, T. (2018). When It Comes to Gorillas, Google Photos Remains Blind www.wir ed.com/story/when-it-comes-to-gorillas-google-photos-remains-blind/ (last accessed April 26, 2020).
Smith, L. N. (2018). “A Disciplined Approach to Neural Network Hyperparameters: Part 1 – Learning Rate, Batch Size, Momentum, and Weight Decay.” http://arxiv.org/abs/1803.09820.
Smith, R. (2007). “An Overview of the Tesseract OCR Engine.” In: Proceedings of the International Conference on Document Analysis and Recognition, ICDAR. Vol. 2, pp. 629–633.
Sobolev, A., et al. (n.d.). “News and Geolocated Social Media Accurately Measure Protest Size.” Cartography and Geographic Information Science 41(3).
Srivastava, N., et al. (2014). “Dropout: A Simple Way to Prevent Neural Networks from Overfitting.” J. Mach. Learn. Res. 15(1), 1929–1958. http://dl.acm.org/citation.cfm?id=2627435.2670313.
Steinert-Threlkeld, Z. C. (2018). Twitter as Data. Elements in Quantitative and Computational Methods for the Social Sciences. Cambridge: Cambridge University Press.
Steinert-Threlkeld, Z. C., and Joo, J (n.d.). “Event Data from Images.”
Suppe, R. (2018). “Orlando Police Decide to Keep Testing Controversial Amazon Facial Recognition Program.” USA Today. www.usatoday.com/story/tech/2018/07/09/orlando-police-decide-keep-testing-amazon-facial-recognition-program/768507002/.
Tai, L., and Liu, M (2016). “Mobile Robots Exploration through CNN- Based Reinforcement Learning.” Robotics and Biomimetics 3(1), 24. https://doi.org/10.1186/s40638-016–0055-x.
Tanksale, N. (2018). Finding Good Learning Rate and The One Cycle Policy. https://towardsdatascience.com/finding-good-learning-rate-and-the-one-cycle-policy-7159fe1db5d6 (last accessed April 26, 2020).
Taylor, L., and Nitschke, G (2017). “Improving Deep Learning using Generic Data Augmentation.” CoRR abs/1708.0. http://arxiv.org/abs/1708.06020.
Tesseract documentation | Tesseract OCR (2020). https://tesseract-ocr.github.io/ (last accessed April 26, 2020).
Todorov, A., et al. (2005). “Inferences of Competence from Faces Predict Election Outcomes.” Science 308(5728), 1623–1626.
torchvision.models – PyTorch Master Documentation (2020). https://pytorch.org/docs/stable/torchvision/models.html (last accessed April 26, 2020).
Torres, M. (2019). “Give Me the Full Picture: Using Computer Vision to Understand Visual Frames and Political Communication.”
University of Oxford (2020). Responsibilities under GDPR | Research Support. https://researchsupport.admin.ox.ac.uk/policy/data/responsibilities (last accessed April 26, 2020).
Webb Williams, N., and Casas, A (2020). “norawebbwilliams/images_as_data: First Release.” Github.
Webb Williams, N., Casas, A, and Wilkerson, J (2020). Images as Data for Social Science Research. https://codeocean.com/capsule/8598 879/tree/v1.
Wickham, H. (2017). tidyverse: Easily Install and Load the “Tidyverse.” https://cran.r-project.org/package=tidyverse.
Williams, M. L., Burnap, P, and Sloan, L (2017). “Towards an Ethical Framework for Publishing Twitter Data in Social Research: Taking into Account Users’ Views, Online Context and Algorithmic Estimation.” Sociology 51(6), 1149–1168.
Wilson, D. R., and Martinez, T. R. (2003). “The General Inefficiency of Batch Training for Gradient Descent Learning.” Neural Networks 16(10), 1429–1451. http://dx.doi.org/10.1016/S0893-6080(03)00138–2.
Won, D., Steinert-Threlkeld, Z. C., and Joo, J (2017). “Protest Activity Detection and Perceived Violence Estimation from Social Media Images.” In: Proceedings of the 25th ACM International Conference on Multimedia. http://arxiv.org/abs/1709.06204.
You, Q. et al. (2015). “Robust Image Sentiment Analysis using Progressively Trained and Domain Transferred Deep Networks.” In: The Twenty-Ninth AAAI Conference, pp. 381–388.
Zhang, H., and Pan, J (2019). “CASM: A Deep Learning Approach for Identifying Collective Action Events with Text and Image Data from Social Media.” Sociological Methodology 49(1), 1–57.
Zhang, Q., and Zhu, S.-C. (2018). “Visual Interpretability for Deep Learning: a Survey.” http://arxiv.org/abs/1802.00614.
Zhao, J., et al. (2017). “Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-Level Constraints.” CoRR abs/1707.0. http://arxiv.org/abs/1707.09457.
Zhu, X., and Ramanan, D (2012). “Face Detection, Pose Estimation, and Landmark Estimation in the Wild.” In: International Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 2879–2886.

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Book summary page views

Total views: 0 *
Loading metrics...

* Views captured on Cambridge Core between #date#. This data will be updated every 24 hours.

Usage data cannot currently be displayed.