Hostname: page-component-5c6d5d7d68-thh2z Total loading time: 0 Render date: 2024-08-17T13:08:53.486Z Has data issue: false hasContentIssue false
  • English
  • Français

Hand gesture recognition method using FMCW radar based on multidomain fusion

Published online by Cambridge University Press:  22 August 2023

Tianhong Yang Open the ORCID record for Tianhong Yang [Opens in a new window]  and
Hanxu Wu
Tianhong Yang*
Affiliation:
College of Electronic Information Engineering, Shenyang Aerospace University, Shenyang, China
Hanxu Wu
Affiliation:
College of Electronic Information Engineering, Shenyang Aerospace University, Shenyang, China
*
Corresponding author: Tianhong Yang; Email: ytianhong@126.com
Get access Rights & Permissions [Opens in a new window]

Abstract

Radar-based hand gesture recognition is a potential noncontact human–machine interaction technique. To enhance the recognition performance of hand gesture, a multidomain fusion-based recognition method using frequency-modulated continuous wave radar is proposed in this article. The received raw echo data of gestures is preprocessed to obtain the range–time matrix, Doppler–time matrix, and range–Doppler–frame tensor. The obtained three-domain radar data corresponding to each gesture are input into the three-channel convolutional neural network for feature extraction. In particular, the extracted features from three-domain data are fused with learnable weight matrices to obtain the final gesture classification results. The experimental results have shown that the classification accuracy of the proposed multidomain fusion network based on learning weight matrix-based fusion is 98.45%, which improves the classification performance compared with the classic average-based fusion and concatenation fusion.

Keywords

convolutional neural networkFrequency modulated continuous wave radarhand gesture recognitionmultidomain fusion
Type
Research Paper
Information
International Journal of Microwave and Wireless Technologies , First View , pp. 1 - 9
Check for updates
Copyright
© The Author(s), 2023. Published by Cambridge University Press in association with The European Microwave Association

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Guo, L, Lu, Z and Yao, L (2021) Human-machine interaction sensing technology based on hand gesture recognition: A review. IEEE Transactions on Human-Machine Systems 51(4), 300–309.CrossRefGoogle Scholar
Plouffe, G and Cretu, M (2016) Static and dynamic hand gesture recognition in depth data using dynamic time warping. IEEE Transactions on Instrumentation and Measurement 65(2), 305–316.CrossRefGoogle Scholar
Li, G, Zhang, R, Ritchie, M and Griffiths, H (2018) Sparsity-driven micro-Doppler feature extraction for dynamic hand gesture recognition. IEEE Transactions on Aerospace and Electronic Systems 54(2), 655–665.CrossRefGoogle Scholar
Khan, F, Leem, SK and Cho, SH (2017) Hand-based gesture recognition for vehicular applications using IR-UWB radar. Sensors 17(4), .CrossRefGoogle ScholarPubMed
Li, Y, Gu, C and Mao, J (2022) 4-D gesture sensing using reconfigurable virtual array based on a 60-GHz FMCW MIMO radar sensor. IEEE Transactions on Microwave Theory and Techniques 70(7), 3652–3665.CrossRefGoogle Scholar
Scherer, M, Magno, M, Erb, J, Mayer, P, Eggimann, M and Benini, L (2021) TinyRadarNN: Combining spatial and temporal convolutional neural networks for embedded gesture recognition with short range radars. IEEE Internet of Things Journal 8(13), 10336–10346.CrossRefGoogle Scholar
Zhang, J, Tao, J and Shi, ZG (2019) Doppler radar-based hand gesture recognition system using convolutional neural networks. In International Conference on Communications, Signal Processing, and Systems, 1096–1113.CrossRefGoogle Scholar
Kim, Y and Toomajian, B (2016) Hand gesture recognition using micro-Doppler signatures with convolutional neural network. IEEE Access 4, 7125–7130.CrossRefGoogle Scholar
Zhang, X, Wu, Q and Zhao, D (2018) Dynamic hand gesture recognition using FMCW radar sensor for driving assistance. In 2018 10th International Conference on Wireless Communications and Signal Processing (WCSP), 1–6.CrossRefGoogle Scholar
Sun, Y, Tai, F, Schliep, F and Pohl, N (2018) Gesture classification with handcrafted micro-Doppler features using a FMCW radar. In 2018 IEEE MTT-S International Conference on Microwaves for Intelligent Mobility (ICMIM), 1–4.CrossRefGoogle Scholar
Zhang, Z, Tian, Z and Zhou, M (2018) Dynamic continuous hand gesture recognition using FMCW radar sensor. IEEE Sensors Journal 18(8), 3278–3289.CrossRefGoogle Scholar
Wang, Y, Shu, Y, Jia, X, Zhou, M, Xie, L and Guo, L (2022) Multifeature fusion-based hand gesture sensing and recognition system. IEEE Geoscience and Remote Sensing Letters 19, 1–5.Google Scholar
Xia, X, Luomei, Y, Zhou, C and Xu, F (2021) Multidimensional feature representation and learning for robust hand-gesture recognition on commercial millimeter-wave radar. IEEE Transactions on Geoscience and Remote Sensing 59(6), 4749–4764.CrossRefGoogle Scholar
Chen, Z, Li, G, Fioranelli, F and Griffiths, H (2019) Dynamic hand gesture classification based on multistatic radar micro-Doppler signatures using convolutional neural network. In IEEE Radar Conference (RadarConf19), 1–5.CrossRefGoogle Scholar
Skaria, S, Hourani, AA and Evans, RJ (2020) Deep-learning methods for hand-gesture recognition using ultra-wideband radar. IEEE Access 8, 203580–203590.CrossRefGoogle Scholar
Lei, W, Jiang, X, Xu, L, Luo, J, Xu, M and Hou, F (2020) Continuous gesture recognition based on time sequence fusion using MIMO radar sensor and deep learning. Electronics 9(5), .CrossRefGoogle Scholar
van der Maaten, L and Hinton, G (2008) Visualizing high-dimensional data using t-SNE. Journal of Machine Learning Research 9(11), 2579–2605.Google Scholar