CASTER: A Computer-Vision-Assisted Wireless Channel Simulator for Gesture Recognition

Zhenyu Ren1, Guoliang Li2, Chenqing Ji1, Chao Yu1, Shuai Wang3, Rui Wang1,*
1Southern University of Science and Technology (SUSTech)
2University of Macau, 3Chinese Academy of Science
*Corresponding author: Prof. Rui Wang (wang.r@sustech.edu.cn)
Paper Code arXiv

Hand gesture channel simulation demo with experiment verification.

Human motion channel simulation demo with experiment verification (Future Work).

Abstract

In this paper, a computer-vision-assisted simulation method is proposed to address the issue of training dataset acquisition for wireless hand gesture recognition. In the existing literature, in order to classify gestures via the wireless channel estimation, massive training samples should be measured in a consistent environment, consuming significant efforts. In the proposed CASTER simulator, however, the training dataset can be simulated via existing videos. Particularly, in the channel simulation, a gesture is represented by a sequence of snapshots, and the channel impulse response of each snapshot is calculated via tracing the rays scattered off a primitive-based hand model. Moreover, CASTER simulator relies on the existing video clips to extract the motion data of gestures. Thus, the massive measurements of wireless channel can be eliminated. The experiments first demonstrate an 83.0% average recognition accuracy of simulation-to-reality inference in recognizing 5 categories of gestures. Moreover, this accuracy can be boosted to 96.5% via the method of transfer learning.

Real-time gesture recognition demo (in Chinese).

Gallery of CASTER.

BibTeX citation


        @article{ren2023caster,
          title={CASTER: A Computer-Vision-Assisted Wireless Channel Simulator for Gesture Recognition},
          author={Ren, Zhenyu and Li, Guoliang and Ji, Chenqing and Yu, Chao and Wang, Shuai and Wang, Rui},
          journal={arXiv preprint arXiv:2311.07169},
          year={2023}
        }