Hostname: page-component-7479d7b7d-jwnkl Total loading time: 0 Render date: 2024-07-12T02:39:00.109Z Has data issue: false hasContentIssue false
  • English
  • Français

An end-to-end neural framework using coarse-to-fine-grained attention for overlapping relational triple extraction

Published online by Cambridge University Press:  21 February 2023

Huizhe Su ,
Hao Wang Open the ORCID record for Hao Wang [Opens in a new window] ,
Xiangfeng Luo  and
Shaorong Xie
Huizhe Su
Affiliation:
School of Computer Engineering and Science, Shanghai University, Shanghai, China
Hao Wang*
Affiliation:
School of Computer Engineering and Science, Shanghai University, Shanghai, China
Xiangfeng Luo
Affiliation:
School of Computer Engineering and Science, Shanghai University, Shanghai, China
Shaorong Xie*
Affiliation:
School of Computer Engineering and Science, Shanghai University, Shanghai, China
*
*Corresponding authors. Email: wang-hao@shu.edu.cn; srxie@shu.edu.cn
*Corresponding authors. Email: wang-hao@shu.edu.cn; srxie@shu.edu.cn
Get access Rights & Permissions [Opens in a new window]

Abstract

In recent years, the extraction of overlapping relations has received great attention in the field of natural language processing (NLP). However, most existing approaches treat relational triples in sentences as isolated, without considering the rich semantic correlations implied in the relational hierarchy. Extracting these overlapping relational triples is challenging, given the overlapping types are various and relatively complex. In addition, these approaches do not highlight the semantic information in the sentence from coarse-grained to fine-grained. In this paper, we propose an end-to-end neural framework based on a decomposition model that incorporates multi-granularity relational features for the extraction of overlapping triples. Our approach employs an attention mechanism that combines relational hierarchy information with multiple granularities and pretrained textual representations, where the relational hierarchies are constructed manually or obtained by unsupervised clustering. We found that the different hierarchy construction strategies have little effect on the final extraction results. Experimental results on two public datasets, NYT and WebNLG, show that our mode substantially outperforms the baseline system in extracting overlapping relational triples, especially for long-tailed relations.

Keywords

Relational extractionRelational hierarchyMulti-granularity informationLong-tail classification
Type
Article
Information
Natural Language Engineering , Volume 29 , Issue 4 , July 2023 , pp. 1126 - 1149
Check for updates
Copyright
© The Author(s), 2023. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Bekoulis, G., Deleu, J., Demeester, T. and Develder, C. (2018). Joint entity recognition and relation extraction as a multi-head selection problem. Expert Systems with Applications 114, 34–45.CrossRefGoogle Scholar
Bi, W. and Kwok, J.T. (2011). Multilabel classification on tree-and dag-structured hierarchies. In Proceedings of the 28th International Conference on Machine Learning, ICML 2011, Bellevue, Washington, USA, June 28 - July 2, 2011, pp. 17–24.Google Scholar
Chen, J., Yuan, C., Wang, X. and Bai, Z. (2019). Mrmep: Joint extraction of multiple relations and multiple entity pairs based on triplet attention. In Proceedings of the 23rd Conference on Computational Natural Language Learning, CoNLL 2019, Hong Kong, China, November 3-4, 2019, pp. 593–602.CrossRefGoogle Scholar
Devlin, J., Chang, M.-W., Lee, K. and Toutanova, K. (2019). Bert: Pre-training of deep bidirectional transformers for language understanding. Association for Computational Linguistics, pp. 4171–4186.Google Scholar
Frey, B.J. and Dueck, D. (2007). Clustering by passing messages between data points. Science 315, 972–976.CrossRefGoogle ScholarPubMed
Fu, T.-J., Li, P.-H. and Ma, W.-Y. (2019). Graphrel: Modeling text as relational graphs for joint entity and relation extraction. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 1409–1418.CrossRefGoogle Scholar
Gardent, C., Shimorina, A., Narayan, S. and Perez-Beltrachini, L. (2017). Creating training corpora for NLG micro-planning. In 55th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 179–188.CrossRefGoogle Scholar
Gormley, R.M., Yu, M. and Dredze, M. (2015). Improved relation extraction with feature-rich compositional embedding models. In Conference on Empirical Methods in Natural Language Processing, pp. 1774–1784.CrossRefGoogle Scholar
Gupta, P., Schütze, H. and Andrassy, B. (2016). Table filling multi-task recurrent neural network for joint entity and relation extraction. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pp. 2537–2547.Google Scholar
Han, X., Yu, P., Liu, Z., Sun, M. and Li, P. (2018). Hierarchical relation extraction with coarse-to-fine grained attention. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2236–2245.CrossRefGoogle Scholar
Hu, Z., Huang, P., Deng, Y., Gao, Y. and Xing, P.E. (2015). Entity hierarchy embedding. The Association for Computer Linguistics, pp. 1292–1300.Google Scholar
Kai, S., Richong, Z., Samuel, M., Yongyi, M. and Xudong, L. (2020). Recurrent interaction network for jointly extracting entities and classifying relations. In EMNLP 2020, pp. 3722–3732.Google Scholar
Li, Q. and Ji, H. (2014). Incremental joint extraction of entity mentions and relations. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 402–412.CrossRefGoogle Scholar
Li, Y., Shen, T., Long, G., Jiang, J., Zhou, T. and Zhang, C. (2020). Improving long-tail relation extraction with collaborating relation-augmented attention. COLING, pp. 1653–1664.Google Scholar
Liang, T., Liu, Y., Liu, X., Sharma, G. and Guo, M. (2021). Distantly-supervised long-tailed relation extraction using constraint graphs. IEEE Transactions on Knowledge and Data Engineering, 1–1.Google Scholar
McCallum, A., Rosenfeld, R., Mitchell, T.M. and Ng, A.Y. (1998). Improving text classification by shrinkage in a hierarchy of classes. ICML 98, 359–367.Google Scholar
Mintz, M., Bills, S., Snow, R. and Jurafsky, D. (2009). Distant supervision for relation extraction without labeled data. Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pp. 1003–1011.CrossRefGoogle Scholar
Miwa, M. and Sasaki, Y. (2014). Modeling joint entity and relation extraction with table representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1858–1869.CrossRefGoogle Scholar
Nayak, T. and Ng, H.T. (2020). Effective modeling of encoder-decoder architecture for joint entity and relation extraction. Proceedings of the AAAI Conference on Artificial Intelligence 34, 8528–8535.CrossRefGoogle Scholar
Ren, X., Wu, Z., He, W., Qu, M., Voss, C.R., Ji, H., Abdelzaher, T.F. and Han, J. (2017). Cotype: Joint extraction of typed entities and relations with knowledge bases. Proceedings of the 26th International Conference on World Wide Web, pp. 1015–1024.CrossRefGoogle Scholar
Riedel, S., Yao, L. and McCallum, A. (2010). Modeling relations and their mentions without labeled text. Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 148–163.CrossRefGoogle Scholar
Rousu, J., Saunders, C., Szedmak, S. and Shawe-Taylor, J. (2005). Learning hierarchical multi-category text classification models. In Proceedings of the 22nd International Conference on Machine Learning, pp. 744–751.CrossRefGoogle Scholar
Takanobu, R., Zhang, T., Liu, J. and Huang, M. (2019). A hierarchical framework for relation extraction with reinforcement learning. In International Conference on Artificial Intelligence, pp. 7072–7079.CrossRefGoogle Scholar
Tipping, M.E. and Bishop, C.M. (1999). Mixtures of probabilistic principal component analysers. Neural Comput 11, 443–482.CrossRefGoogle Scholar
Verma, N., Mahajan, D., Sellamanickam, S. and Nair, V. (2012). Learning hierarchical similarity metrics. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2280–2287.CrossRefGoogle Scholar
Wang, Y., Yu, B., Zhang, Y., Liu, T., Zhu, H. and Sun, L. (2020). Tplinker: Single-stage joint extraction of entities and relations through token pair linking. In COLING, pp. 1572–1582.CrossRefGoogle Scholar
Wei, Z., Su, J., Wang, Y., Tian, Y. and Chang, Y. (2020). A novel cascade binary tagging framework for relational triple extraction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pp. 1476–1488.CrossRefGoogle Scholar
Weinberger, K.Q. and Chapelle, O. (2009). Large margin taxonomy embedding for document categorization. In Advances in Neural Information Processing Systems, pp. 1737–1744.Google Scholar
Xiao, L., Zhou, D. and Wu, M. (2011). Hierarchical classification via orthogonal transfer. In Proceedings of the 28th International Conference on Machine Learning, ICML 2011, Bellevue, Washington, USA, June 28 - July 2, 2011, pp. 801–808.Google Scholar
Xie, R., Liu, Z. and Sun, M. (2016). Representation learning of knowledge graphs with hierarchical types. IJCAI, 2965–2971.Google Scholar
Ye, H., Zhang, N., Deng, S., Chen, M., Tan, C., Huang, F. and Chen, H. (2021). Contrastive triple extraction with generative transformer. Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, pp. 14257–14265.CrossRefGoogle Scholar
Yu, B., Zhang, Z., Shu, X., Wang, Y., Liu, T., Wang, B. and Li, S. (2020). Joint extraction of entities and relations based on a novel decomposition strategy. In ECAI 2020 - 24th European Conference on Artificial Intelligence, 29 August-8 September 2020, Santiago de Compostela, Spain, August 29 - September 8, 2020 - Including 10th Conference on Prestigious Applications of Artificial Intelligence (PAIS 2020), 325:2282–2289.Google Scholar
Yu, X. and Lam, W. (2010). Jointly identifying entities and extracting relations in encyclopedia text via a graphical model approach. Coling 2010: Posters, pp. 1399–1407.Google Scholar
Yuan, Y., Zhou, X., Pan, S., Zhu, Q., Song, Z. and Guo, L. (2020). A relation-specific attention network for joint entity and relation extraction. IJCAI 2020, pp. 4054–4060.CrossRefGoogle Scholar
Zeng, D., Zhang, H. and Liu, Q. (2020). Copymtl: Copy mechanism for joint extraction of entities and relations with multi-task learning. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05):9507–9514.Google Scholar
Zeng, X., He, S., Zeng, D., Liu, K., Liu, S. and Zhao, J. (2019). Learning the extraction order of multiple relational facts in a sentence with reinforcement learning. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 367–377.CrossRefGoogle Scholar
Zeng, X., Zeng, D., He, S., Liu, K. and Zhao, J. (2018). Extracting relational facts by an end-to-end neural model with copy mechanism. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 506–514.CrossRefGoogle Scholar
Zhang, N., Deng, S., Sun, Z., Wang, G., Chen, X., Zhang, W. and Chen, H. (2019). Long-tail relation extraction via knowledge graph embeddings and graph convolution networks. Association for Computational Linguistics, 3016–3025.Google Scholar
Zhang, N., Deng, S., Ye, H., Zhang, W. and Chen, H. (2022). Robust triple extraction with cascade bidirectional capsule network. Expert Systems With Applications, 187, 115806.CrossRefGoogle Scholar
Zhang, N., Ye, H., Deng, S., Tan, C., Chen, M., Huang, S., Huang, F. and Chen, H. (2021). Contrastive information extraction with generative transformer. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 3077–3088.CrossRefGoogle Scholar
Zhang, Z., Zhuang, F., Qu, M., Lin, F. and He, Q. (2018). Knowledge graph embedding with hierarchical relation structure. In EMNLP, 3198–3207.CrossRefGoogle Scholar
Zhao, B., Li, F.-F. and Xing, P.E. (2011). Large-scale category structure aware image categorization. In NIPS, 1251–1259.Google Scholar
Zheng, H., Wen, R., Chen, X., Yang, Y., Zhang, Y., Zhang, Z., Zhang, N., Qin, B., Xu, M. and Zheng, Y. (2021). PRGC: Potential relation and global correspondence based joint relational triple extraction. In Annual Meeting of the Association for Computational Linguistics, 6225–6235.CrossRefGoogle Scholar
Zheng, S., Wang, F., Bao, H., Hao, Y., Zhou, P. and Xu, B. (2017). Joint extraction of entities and relations based on a novel tagging scheme. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers, pp. 1227–1236.CrossRefGoogle Scholar