Forked Recurrent Neural Network for Hand Gesture Classification Using Inertial Measurement Data

Philipp Koch, Nele Brugge, Huy Phan, Marco Maass, Alfred Mertins


For many applications of hand gesture recognition, a delay-free, affordable, and mobile system relying on body signals is mandatory. Therefore, we propose an approach for hand gestures classification given signals of inertial measurement units (IMUs) that works with extremely short windows to avoid delays. With a simple recurrent neural network the suitability of the sensor modalities of an IMU (accelerometer, gyroscope, magnetometer) are evaluated by only providing data of one modality. For the multi-modal data a second network with mid-level fusion is proposed. Its forked architecture allows us to process data of each modality individually before carrying out a joint analysis for classification. Experiments on three databases reveal that even when relying on a single modality our proposed system outperforms state-of-the-art systems significantly. With the forked network classification accuracy can be further improved by over 10 % absolute compared to the best reported system while causing a fraction of the delay.

Original languageEnglish
Title of host publicationICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Number of pages5
Publication date05.2019
Article number8682986
ISBN (Print)978-1-4799-8132-8
ISBN (Electronic)978-1-4799-8131-1
Publication statusPublished - 05.2019
Event44th IEEE International Conference on Acoustics, Speech, and Signal Processing - Brighton Conference Centre, Brighton, United Kingdom
Duration: 12.05.201917.05.2019
Conference number: 149034


Dive into the research topics of 'Forked Recurrent Neural Network for Hand Gesture Classification Using Inertial Measurement Data'. Together they form a unique fingerprint.

Cite this