Ultrasound-EMG combined hand gesture recognition
From iis-projects
Contents
Short Description
Electromyography (EMG) is the golden standard offering high accuracy in hand gesture recognition. However, its performance for regression tasks such as predicting the continuous hand and wrist kinematics is much poorer. At the same time, EMG fails in detecting some specific arm movements (e.g., wrist rotations). Combining EMG with other sensing modalities capable of directly inferring information from deep muscular structures would allow to improve the hand gesture recognition task.
In this context, Ultrasound (US) appears as a promising candidate for this task. US imaging is a non-invasive imaging technique that provides visible information on the structure of musculoskeletal tissues, organs, and vascular system. Recent research trends include the development of wearable US probes, with minimal power consumption and reduced count of piezoelectric elements. In fact, meaningful physiological information can be derived also from single-element transducers. Within this framework, at IIS we have recently developed an ultra-low power wearable US probe, based on an MSP430 and nRF52 microcontrollers, operating with a single US channel and consuming less than 20mW [1].
This project tackles the goal of combining EMG and US for gesture recognitions, by designing a custom armband that merges together these two modalities.
Goal & Tasks
In this project, you will work with a novel wearable US probe [1] and ExG probe [2] to perform hand gesture recognitions. The main tasks are:
- design of the armband, accommodating ultrasound transducers and EMG electrodes
- possible design of PCB to adapt the US-EMG readout platform based on existing devices developed at IIS (WULPUS, BIOGAP)
- experiments / data collection of a dataset combining EMG and US
- data analysis / design of appropriate ML models inspired by existing US/EMG feature extraction algorithms developed at IIS (and beyond)
- deployment of the selected ML model on the armband for online operation
Literature
Prerequisites
- embedded C
- Python
- basic Machine Learning
Status: In Progress
Giusy Spacone
- Supervision: Sergei Vostrikov, Victor Kartsch Andrea Cossettini
Character
- 10% Literature Study
- 20% data collection and analysis
- 40% algorithm development
- 30% system development
