Personal tools

Neural Network Algorithms and Interfaces with Accelerators for Embedded Platforms with Real World Applications

From iis-projects

Jump to: navigation, search

Short Description

Neural networks and other machine learning approaches accurately predict patterns in image and time-based data when large historical datasets with curated datasets, excessive compute power including GPUs, and large amounts of time are available. For most applications these prerequisites are not given, and it is important to achieve good predictive results for models that are quickly adapted from a standard pattern to an individual personalized pattern.

IBM Research Zurich has an opening for a student project (preferably Master Thesis, but also Semester Project can be considered) with the objective of achieving an usable workflow for neural network personalization to be used for monitoring activity of daily living of elderly and to evaluate psychological and physiological stress in firemen by using leading hardware (HW) accelerators.

We propose three different HW platforms: Cloud field programmable gate arrays (FPGA, tensor processing units (TPU), and PULP-based platforms ( The FPGA accelerator service is accessed via an application program interface (API) and a scheduler. The front-end API receives accelerator service calls through the OpenStack dashboard or a command-line interface. The Coral Dev Board, a compact board with an edge tensor processing unit (TPU) AI accelerator chip speeds up machine learning. A new family of classification models -- Net-EdgeTPU identifies relationships among a baseline AI model's scaling dimensions under a fixed resource constraint ( The appropriate scaling coefficients for each dimension are then applied to scale up the model to the desired size or computational budget. The PULP platform is an open hardware multi-core platform achieving leading-edge energy-efficiency and featuring widely-tunable performance enabling battery-operated artificial intelligence (AI) in Internet of Things (IoT) applications. It is an open source platform and it comes with an SDK available on GitHub (

The project (Master Thesis) will take place at the IBM Research, Zurich.

Application Scenarios

One application is stress-detection from smart-device data, for activities in extreme environments, ( Stress is a root cause for many modern, chronic diseases, thus wearble stress monitoring has a huge potential for stress prevention and management and to improve the quality of life. However, mental stress also critically affects decision making skills. Thus, tools to detect mental stress early, can significantly contribute to work safety in extreme conditions. Traditional stress detection methods are not practical for field-deployment. However, with the availability of low-cost consumer wearable devices that monitor vital signs in real time, more practical stress detection schemes have become possible. Previously, IBM Research measured heart rate variability (HRV) with wearable devices in realistic training environments for firefighters, who were subject to physical, psychological and combined stress. Using machine learning algorithms, different stress types were identified with 88% accuracy, in 1-minute time windows. These predictions are used to help firefighters train more efficiently and experience personal limits, help coordinators to put together the right team for specific missions and finally help mission commanders keep their teams safe. If combined with AI at the edge, to additionally extract context information, real-time closed loop risk mitigation schemes can be implemented with a few seconds latencies. The method is not limited to stress monitoring but can be extended to activity monitoring of daily living of elderly ( While activity of daily living personalization and analysis is not as time critical, future expansions, for example fall detection requires faster response times profit from edge acceleration.

Goal & Tasks

Building on the existing model and data provided by IBM, in this project you will:

  • Literature study of relevant background context
  • Achieve a usable workflow for neural network personalization
  • Derive new algorithms to improve performance, also in view of running them on performance constraint embedded platforms
  • Help complete the current dataset acquisition for quantitative stress assessment with additional vital signs like motion or audio and carry out the initial data-analysis
  • Data curation
  • With the new data, assess applicability and portability of personalized deep learning models to generic “community” models and their dependence on personal baselines and expand the model towards federate learning schemes


(not all need to be met by the single candidate)

  • Knowdleg of Python and some cloud computing
  • Knowledge of machine learning and signal processing
  • Familiarity with deep learning frameworks (Keras, TensorFlow)
  • Motivation to build and test a real system and acquiring field data

Detailed Task Description

A detailed task description will be worked out right before the project, taking the student's interests and capabilities into account.

Status: Available

  • Looking for Semester and Master Project Students
Supervisors: Michele Magno, Xiaying Wang
IBM contacts: Dr. Bruno Michel, Dr. Jonas Weiss


40% Theory and Algorithms
40% Implementation
20% Verification and Testing

IIS Professor

Luca Benini

↑ top

Practical Details