Hostname: page-component-84b7d79bbc-2l2gl Total loading time: 0 Render date: 2024-07-29T03:23:28.344Z Has data issue: false hasContentIssue false
  • English
  • Français

Aspect opinion expression and rating prediction via LDA–CRF hybrid

Published online by Cambridge University Press:  22 April 2018

ABHISHEK LADDHA Open the ORCID record for ABHISHEK LADDHA [Opens in a new window]  and
ARJUN MUKHERJEE
ABHISHEK LADDHA
Affiliation:
Indian Institute of Technology, Delhi, New Delhi, India e-mail: laddhaabhishek11@gmail.com
ARJUN MUKHERJEE
Affiliation:
Department of Computer Science, University of Houston, Houston, Texas, USA e-mail: arjun@cs.uh.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

In this paper, we study the problem of aspect-based sentiment analysis. Our model simultaneously extracts aspect-specific opinion expressions and determines the rating for each aspect in reviews. Previous works have mainly focused on the problem of opinion phrase extraction and aspect rating prediction in a pipelined manner and are not able to capture the dependencies of aspect opinion expression on aspect rating and vice-versa. They are also unable to discover aspect-specific opinion expressions and their associated rating scores. We present a joint modelling approach to extract aspect-specific sentiment expression and aspect rating prediction simultaneously. This paper proposes a novel LDA–CRF hybrid model which employs discriminative conditional random field component for phrase extraction, a regression component for rating prediction and a generative component for grouping aspect–sentiment expressions (aspect-specific opinion expressions) into coherent topics. To show the effectiveness of our approach, we evaluate the performance of the model on both task: (i) aspect-specific opinion expressions and (ii) rating prediction on the dataset of hotel and restaurant reviews from TripAdvisor.com. Experimental results show that both task potentially reinforce each other and joint modeling outperformed state-of-the-art baselines for each individual tasks.

Type
Article
Information
Natural Language Engineering , Volume 24 , Issue 4 , July 2018 , pp. 611 - 639
Copyright
Copyright © Cambridge University Press 2018 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Bengio, S., Vinyals, O., Jaitly, N., and Shazeer, N. 2015. Scheduled sampling for sequence prediction with recurrent neural networks. In Proceedings of the 28th International Conference on Advances in Neural Information Processing Systems, pp. 1171–9.Google Scholar
Breck, E., Choi, Y., and Cardie, C. 2007. Identifying expressions of opinion in context. In IJCAI, vol. 7, pp. 2683–8.Google Scholar
Brody, S., and Elhadad, N. 2010. An unsupervised aspect-sentiment model for online reviews. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Association for Computational Linguistics, pp. 804–12.Google Scholar
Chang, C.-C., and Lin, C.-J. 2011. LIBSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2. http://www.csie.ntu.edu.tw/~cjlin/libsvm.CrossRefGoogle Scholar
Choi, Y., Breck, E., and Cardie, C. 2006. Joint extraction of entities and relations for opinion recognition. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, pp. 431–9.Google Scholar
Choi, Y., Cardie, C., Riloff, E., and Patwardhan, S. 2005. Identifying sources of opinions with conditional random fields and extraction patterns. In Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing, Association for Computational Linguistics, pp. 355–62.Google Scholar
Dave, K., Lawrence, S., and Pennock, D. M. 2003. Mining the peanut gallery: opinion extraction and semantic classification of product reviews. In Proceedings of the 12th International Conference on World Wide Web, ACM, pp. 519–28.Google Scholar
Diao, Q., Qiu, M., Wu, C.-Y., Smola, A. J., Jiang, J., and Wang, C. 2014. Jointly modeling aspects, ratings and sentiments for movie recommendation (jmars). In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, pp. 193–202.Google Scholar
Diebolt, J., and Ip, E. 1996. Markov chain monte carlo in practice.Google Scholar
El-Kishky, A., Song, Y., Wang, C., Voss, C. R., and Han, J., 2014. Scalable topical phrase mining from text corpora. Proceedings of the VLDB Endowment 8 (3): 305–16.CrossRefGoogle Scholar
Fei, G., Chen, Z., and Liu, B. 2014. Review topic discovery with phrases using the pólya urn model. In COLING, pp. 667–76.Google Scholar
Fu, X., Wu, H., and Cui, L. 2016. Topic sentiment joint model with word embeddings. In DMNLP@ PKDD/ECML, pp. 41–8.Google Scholar
Fu, X., Yang, K., Huang, J. Z., and Cui, L., 2015. Dynamic non-parametric joint sentiment topic mixture model. Knowledge-Based Systems 82 : 102–14.CrossRefGoogle Scholar
Hu, M., and Liu, B. 2004. Mining and summarizing customer reviews. In Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, pp. 168–77.Google Scholar
Huang, Z., Xu, W., and Yu, K. 2015. Bidirectional lstm-crf models for sequence tagging. arXiv preprint, arXiv:1508.01991.Google Scholar
Irsoy, O., and Cardie, C. 2014. Opinion mining with deep recurrent neural networks. In EMNLP, pp. 720–8.Google Scholar
Jakob, N., and Gurevych, I. 2010. Extracting opinion targets in a single-and cross-domain setting with conditional random fields. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, pp. 1035–45.Google Scholar
Jo, Y. and Oh, A. H. 2011. Aspect and sentiment unification model for online review analysis. In Proceedings of the 4th ACM International Conference on Web Search and Data Mining, ACM, pp. 815–24.Google Scholar
Johansson, R., and Moschitti, A., 2011. Extracting opinion expressions and their polarities: exploration of pipelines and joint models. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers, vol. 2, Association for Computational Linguistics, pp. 101–6.Google Scholar
Kim, S.-M., and Hovy, E. 2006. Extracting opinions, opinion holders, and topics expressed in online news media text. In Proceedings of the Workshop on Sentiment and Subjectivity in Text, Association for Computational Linguistics, pp. 1–8.Google Scholar
Kobayashi, N., Inui, K., and Matsumoto, Y., 2007. Extracting aspect-evaluation and aspect-of relations in opinion mining. In EMNLP-CoNLL, vol. 7, Citeseer, pp. 1065–74.Google Scholar
Kudo, T. 2009. Crf++: Yet another crf toolkit [ol].Google Scholar
Lafferty, J., McCallum, A., and Pereira, F. 2001. Conditional random fields: probabilistic models for segmenting and labeling sequence data. In Proceedings of the 18th International Conference on Machine Learning, ICML, vol. 1, pp. 282–9.Google Scholar
Lakkaraju, H., Bhattacharyya, C., Bhattacharya, I., and Merugu, S. 2011. Exploiting coherence for the simultaneous discovery of latent facets and associated sentiments. In SDM, SIAM, pp. 498–509.Google Scholar
Lakkaraju, H., Socher, R., and Manning, C. 2014. Aspect specific sentiment analysis using hierarchical deep learning. In NIPS Workshop on Deep Learning and Representation Learning.Google Scholar
Li, F., Huang, M., and Zhu, X. 2010. Sentiment analysis with global topics and local dependency. In AAAI, vol. 10, pp. 1371–6.Google Scholar
Lin, C., and He, Y. 2009. Joint sentiment/topic model for sentiment analysis. In Proceedings of the 18th ACM Conference on Information and Lnowledge Management, ACM, pp. 375–84.Google Scholar
Lindsey, R. V., Headden, W. P. III, and Stipicevic, M. J. 2012. A phrase-discovering topic model using hierarchical pitman-yor processes. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, Association for Computational Linguistics, pp. 214–22.Google Scholar
Liu, B. and Zhang, L. 2012. A survey of opinion mining and sentiment analysis. In Mining Text Data, pp. 415–63. Springer, pp. 415–63.Google Scholar
Liu, J., Shang, J., Wang, C., Ren, X., and Han, J. 2015. Mining quality phrases from massive text corpora. In Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, ACM, pp. 1729–44.Google Scholar
Liu, P., Joty, S. R., and Meng, H. M. 2015. Fine-grained opinion mining with recurrent neural networks and word embeddings. In EMNLP, pp. 1433–43.Google Scholar
Lu, B., Ott, M., Cardie, C., and Tsou, B. K. 2011. Multi-aspect sentiment analysis with topic models. In Proceedings of the IEEE 11th International Conference on Data Mining Workshops, IEEE, pp. 81–8.Google Scholar
Lu, Y., Zhai, C., and Sundaresan, N. 2009. Rated aspect summarization of short comments. In Proceedings of the 18th International Conference on World Wide Web, ACM, pp. 131–40.Google Scholar
Mahmoud, H. 2008. Pólya urn Models. CRC press.CrossRefGoogle Scholar
Mcauliffe, J. D., and Blei, D. M. 2008. Supervised topic models. In Advances in Neural Information Processing Systems, pp. 121–8.Google Scholar
Mei, Q., Ling, X., Wondra, M., Su, H., and Zhai, C. 2007. Topic sentiment mixture: modeling facets and opinions in weblogs. In Proceedings of the 16th International Conference on World Wide Web, ACM, pp. 171–80.Google Scholar
Mimno, D. and McCallum, A. 2012. Topic models conditioned on arbitrary features with dirichlet-multinomial regression. arXiv preprint, arXiv:1206.3278.Google Scholar
Mimno, D., Wallach, H. M., Talley, E., Leenders, M., and McCallum, A. 2011. Optimizing semantic coherence in topic models. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, pp. 262–72.Google Scholar
Moghaddam, S., and Ester, M. 2011. Ilda: interdependent lda model for learning latent aspects and their ratings from online product reviews. In Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, ACM, pp. 665–74.Google Scholar
Moghaddam, S., and Ester, M. 2013. The flda model for aspect-based opinion mining: addressing the cold start problem. In Proceedings of the 22nd International Conference on World Wide Web, International World Wide Web Conferences Steering Committee, pp. 909–18.Google Scholar
Mukherjee, A., and Liu, B., 2012. Aspect extraction through semi-supervised modeling. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers, vol. 1, Association for Computational Linguistics, pp. 339–48.Google Scholar
Nguyen, T. H., and Shirai, K. 2015. Phrasernn: phrase recursive neural network for aspect-based sentiment analysis. In EMNLP, pp. 2509–14.CrossRefGoogle Scholar
Pennington, J., Socher, R., and Manning, C. D. 2014. Glove: global vectors for word representation. In EMNLP, vol. 14, pp. 1532–43.Google Scholar
Pontiki, M., et al. 2016. Semeval-2016 task 5: aspect based sentiment analysis. In ProWorkshop on Semantic Evaluation (SemEval-2016), Association for Computational Linguistics, pp. 19–30.Google Scholar
Popescu, A.-M., and Etzioni, O. 2007. Extracting product features and opinions from reviews. In Natural Language Processing and Text Mining, Springer, pp. 9–28.Google Scholar
Poria, S., Cambria, E., and Gelbukh, A. 2016. Aspect extraction for opinion mining with a deep convolutional neural network. Knowledge-Based Systems 108, 42–9.Google Scholar
Poria, S., Chaturvedi, I., Cambria, E., and Bisio, F. 2016. Sentic lda: improving on lda with semantic similarity for aspect-based sentiment analysis. In Proceedings of the International Joint Conference on Neural Networks (IJCNN), pp. 4465–73.Google Scholar
Sauper, C., and Barzilay, R., 2013. Automatic aggregation by joint modeling of aspects and values. Journal of Artificial Intelligence Research 46 : 89–127.Google Scholar
Sauper, C., Haghighi, A., and Barzilay, R., 2011. Content models with attitude. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, vol. 1, Association for Computational Linguistics, pp. 350–8.Google Scholar
Snyder, B., and Barzilay, R. 2007. Multiple aspect ranking using the good grief algorithm. In HLT-NAACL, pp. 300–7.Google Scholar
Somasundaran, S., Namata, G., Wiebe, J., and Getoor, L., 2009. Supervised and unsupervised methods in employing discourse relations for improving opinion polarity classification. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, vol. 1, Association for Computational Linguistics, pp. 170–9.Google Scholar
Tang, D., Qin, B., Feng, X., and Liu, T. 2015. Effective lstms for target-dependent sentiment classification. arXiv preprint, arXiv:1512.01100.Google Scholar
Tang, D., Qin, B., and Liu, T. 2016. Aspect level sentiment classification with deep memory network. arXiv preprint, arXiv:1605.08900.Google Scholar
Titov, I., and McDonald, R. T., 2008. A joint model of text and aspect ratings for sentiment summarization. In ACL, vol. 8, Citeseer, pp. 308–16.Google Scholar
Wang, H. 2015. Sentiment-Aligned Topic Models for Product Aspect Rating Prediction. Ph D Thesis, Applied Sciences, School of Computing Science.CrossRefGoogle Scholar
Wang, H., Lu, Y., and Zhai, C. 2010. Latent aspect rating analysis on review text data: a rating regression approach. In Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, pp. 783–92.Google Scholar
Wang, H., Lu, Y., and Zhai, C. 2011. Latent aspect rating analysis without aspect keyword supervision. In Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, pp. 618–26.Google Scholar
Wang, W., Pan, S. J., Dahlmeier, D., and Xiao, X. 2016. Recursive neural conditional random fields for aspect-based sentiment analysis. arXiv preprint, arXiv:1603.06679.Google Scholar
Wang, X., McCallum, A., and Wei, X. 2007. Topical n-grams: phrase and topic discovery, with an application to information retrieval. In Proceedings of the 7th IEEE International Conference on Data Mining, ICDM 2007, pp. 697–702.Google Scholar
Wang, Y., Huang, M., Zhao, L., and Zhu, X. 2016. Attention-based lstm for aspect-level sentiment classification. In EMNLP, pp. 606–15.Google Scholar
Wiebe, J., Wilson, T., and Cardie, C., 2005. Annotating expressions of opinions and emotions in language. Language Resources and Evaluation 39 (2–3): 165–210.CrossRefGoogle Scholar
Wu, Y., Zhang, Q., Huang, X., and Wu, L., 2009. Phrase dependency parsing for opinion mining. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, vol. 3, Association for Computational Linguistics, pp. 1533–41.Google Scholar
Xianghua, F., Guo, L., Yanyan, G., and Zhiqiang, W., 2013. Multi-aspect sentiment analysis for chinese online social reviews based on topic modeling and hownet lexicon. Knowledge-Based Systems 37 : 186–95.CrossRefGoogle Scholar
Xu, Y., Lin, T., Lam, W., Zhou, Z., Cheng, H., and So, A. M.-C. 2014. Latent aspect mining via exploring sparsity and intrinsic information. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, ACM, pp. 879–88.Google Scholar
Yang, B., and Cardie, C. 2012. Extracting opinion expressions with semi-markov conditional random fields. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, Association for Computational Linguistics, pp. 1335–45.Google Scholar
Yang, B., and Cardie, C. 2013. Joint inference for fine-grained opinion extraction. In ACL (1), pp. 1640–9.Google Scholar
Yang, B., and Cardie, C., 2014. Joint modeling of opinion expression extraction and attribute classification. Transactions of the Association for Computational Linguistics 2 : 505–16.CrossRefGoogle Scholar
Yin, Y., Wei, F., Dong, L., Xu, K., Zhang, M., and Zhou, M. 2016. Unsupervised word and dependency path embeddings for aspect term extraction. arXiv preprint, arXiv:1605.07843.Google Scholar
Zhao, W. X., Jiang, J., He, J., Song, Y., Achananuparp, P., Lim, E.-P., and Li, X., 2011. Topical keyphrase extraction from twitter. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, vol. 1, Association for Computational Linguistics, pp. 379–88.Google Scholar
Zhao, W. X., Jiang, J., Yan, H., and Li, X. 2010. Jointly modeling aspects and opinions with a maxent-lda hybrid. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, pp. 56–65.Google Scholar
Zhu, J., Ahmed, A., and Xing, E. P., 2012. Medlda: maximum margin supervised topic models. Journal of Machine Learning Research 13 : 2237–78.Google Scholar
Zhuang, L., Jing, F., and Zhu, X.-Y. 2006. Movie review mining and summarization. In Proceedings of the 15th ACM International Conference on Information and Knowledge Management, ACM, pp. 43–50.Google Scholar