Personal tools

Difference between revisions of "Neural Network Algorithms and Interfaces with Accelerators for Embedded Platforms with Real World Applications"

From iis-projects

Jump to: navigation, search
(Application Scenario)
 
Line 2: Line 2:
 
==Short Description==
 
==Short Description==
  
Neural networks and other machine learning approaches accurately predict patterns in image and time-based data when large historical datasets, excessive compute power including GPUs, and large amounts of time are available. For most applications these prerequisites are not given, and it is important to achieve good predictive results for models that are quickly adapted from a standard pattern to an individual personalized pattern.
+
Neural networks and other machine learning approaches accurately predict patterns in image and time-based data when large historical datasets with curated datasets, excessive compute power including GPUs, and large amounts of time are available. For most applications these prerequisites are not given, and it is important to achieve good predictive results for models that are quickly adapted from a standard pattern to an individual personalized pattern.
  
IBM Research Zurich has an opening for a student project (preferably Master Thesis, but also Semester Project can be considered) with the objective of achieving an usable workflow for neural network personalization to be used for monitoring actibity of daily libing of elderly and to evaluate psychological and physiological stress in firemen by using leading hardware (HW) accelerators. We propose three different HWs: cloud field programmable gate arrays (FPGA https://www.zurich.ibm.com/cci/cloudFPGA/), tensor processing units (TPU), and PULP-based platforms (https://pulp-platform.org). The FPGA accelerator service is accessed via an application program interface (API) and a scheduler. The front-end API receives accelerator service calls through the OpenStack dashboard or a command-line interface. The Coral Dev Board, a compact PC featuring a tensor processing unit (Edge TPU) AI accelerator chip speeds up machine learning inference. A new family of classification models -- EfficientNet-EdgeTPU use a grid serach to identify relationships among a baseline AI model's scaling dimensions under a fixed resource constraint. The search determines the appropriate scaling coefficient for each dimension, and the coefficients are then applied to scale up the baseline model to the desired model size or computational budget. The PULP platform is an open hardware multi-core platform achieving leading-edge energy-efficiency and featuring widely-tunable performance enabling battery-operated artificial intelligence (AI) in Internet of Things (IoT) applications.
+
IBM Research Zurich has an opening for a student project (preferably Master Thesis, but also Semester Project can be considered) with the objective of achieving an usable workflow for neural network personalization to be used for monitoring activity of daily living of elderly and to evaluate psychological and physiological stress in firemen by using leading hardware (HW) accelerators.
 +
 
 +
We propose three different HW platforms: Cloud field programmable gate arrays (FPGA https://www.zurich.ibm.com/cci/cloudFPGA/), tensor processing units (TPU), and PULP-based platforms (https://pulp-platform.org). The FPGA accelerator service is accessed via an application program interface (API) and a scheduler. The front-end API receives accelerator service calls through the OpenStack dashboard or a command-line interface. The Coral Dev Board, a compact board with an edge tensor processing unit (TPU) AI accelerator chip speeds up machine learning. A new family of classification models -- Net-EdgeTPU identifies relationships among a baseline AI model's scaling dimensions under a fixed resource constraint (https://coral.withgoogle.com/docs/edgetpu/benchmarks/). The appropriate scaling coefficients for each dimension are then applied to scale up the model to the desired size or computational budget. The PULP platform is an open hardware multi-core platform achieving leading-edge energy-efficiency and featuring widely-tunable performance enabling battery-operated artificial intelligence (AI) in Internet of Things (IoT) applications. It is an open source platform and it comes with an SDK available on GitHub (https://github.com/pulp-platform/pulp-sdk).  
  
 
The project (Master Thesis) will take place at the IBM Research, Zurich.
 
The project (Master Thesis) will take place at the IBM Research, Zurich.
  
 
==Application Scenarios==
 
==Application Scenarios==
One possible application is the stress-detection and -interpretation from smart-device data, for activities in extreme environments, such as oil-rig workers, construction divers or firefighters (https://researcher.watson.ibm.com/researcher/view_group.php?id=10009).
+
One application is stress-detection from smart-device data, for activities in extreme environments, (https://researcher.watson.ibm.com/researcher/view_group.php?id=10009).
 
+
Stress is a root cause for many modern, chronic diseases, thus wearble stress monitoring has a huge potential for stress prevention and management and to improve the quality of life. However, mental stress also critically affects decision making skills. Thus, tools to detect mental stress early, can significantly contribute to work safety in extreme conditions.
Stress has been identified as the root cause for many modern, chronic diseases, thus has a huge potential for prevention and management of the latter and to improve the quality of life. However, mental stress also critically affects decision making skills. Thus, tools to detect mental stress early, can significantly contribute to work safety in many work environments, particularly under extreme conditions.
+
Traditional stress detection methods are not practical for field-deployment. However, with the availability of low-cost consumer wearable devices that monitor vital signs in real time, more practical stress detection schemes have become possible. Previously, IBM Research measured heart rate variability (HRV) with wearable devices in realistic training environments for firefighters, who were subject to physical, psychological and combined stress. Using machine learning algorithms, different stress types were identified with 88% accuracy, in 1-minute time windows. These predictions are used to help firefighters train more efficiently and experience personal limits, help coordinators to put together the right team for specific missions and finally help mission commanders keep their teams safe. If combined with AI at the edge, to additionally extract context information, real-time closed loop risk mitigation schemes can be implemented with a few seconds latencies. The method is not limited to stress monitoring but can be extended to activity monitoring of daily living of elderly (https://www.activageproject.eu/deploymentsites/Region-Emilia-Romagna/). While activity of daily living personalization and analysis is not as time critical, future expansions, for example fall detection requires faster response times profit from edge acceleration.
 
 
Traditional stress detection methods tend to be cumbersome or not practical for field-deployment at all. However, with the availability of low-cost consumer wearable devices that monitor vital signs in real time, more practical stress detection schemes have become possible.
 
 
 
In previous works, IBM Research Zurich derived the heart rate variability (HRV), a stress-related vital-sign, from wearable device data that are collected in a controlled, but realistic environment for firefighters, who were subject to physical, psychological and combined stress. Using machine learning algorithms, different stress types are identified with 88% accuracy, in 1-minute time windows. These predictions are used to help firefighters train more efficiently and experience personal limits, help coordinators to put together the right team for specific missions and finally help mission commanders keep their teams safe in the field. If combined with AI to additionally extract context information, real-time closed loop risk mitigation schemes can be implemented with a few seconds latencies, as required for firefighters.
 
 
 
Another application scenario is the activity monitorig of daily libing of elderly (https://www.activageproject.eu/deployment-sites/Region-Emilia-Romagna/).
 
  
 
===Goal & Tasks===
 
===Goal & Tasks===
 
Building on the existing model and data provided by IBM, in this project you will:
 
Building on the existing model and data provided by IBM, in this project you will:
 
* Literature study of relevant background context
 
* Literature study of relevant background context
*      Achieve an usable workflow for neural network personalization
+
*      Achieve a usable workflow for neural network personalization
 
* Derive new algorithms to improve performance, also in view of running them on performance constraint embedded platforms
 
* Derive new algorithms to improve performance, also in view of running them on performance constraint embedded platforms
 
* Help complete the current dataset acquisition for quantitative stress assessment with additional vital signs like motion or audio and carry out the initial data-analysis
 
* Help complete the current dataset acquisition for quantitative stress assessment with additional vital signs like motion or audio and carry out the initial data-analysis
 +
*      Data curation
 
* With the new data, assess applicability and portability of personalized deep learning models to generic “community” models and their dependence on personal baselines and expand the model towards federate learning schemes
 
* With the new data, assess applicability and portability of personalized deep learning models to generic “community” models and their dependence on personal baselines and expand the model towards federate learning schemes
  
 
===Prerequisites===
 
===Prerequisites===
 
(''not all need to be met'' by the single candidate)
 
(''not all need to be met'' by the single candidate)
* Knowledge of embedded system programming and PC programming (C/C++, preferably embedded C, HDL)
+
* Knowdleg of Python and some cloud computing
 
* Knowledge of machine learning and signal processing
 
* Knowledge of machine learning and signal processing
* Motivation to build and test a real system and acquiring dataset
+
*      Familiarity with deep learning frameworks (Keras, TensorFlow)
*      PCB Desing or willing to learn it (only for students interested in system design)
+
* Motivation to build and test a real system and acquiring field data
  
  

Latest revision as of 17:29, 16 September 2019

Applications.png

Short Description

Neural networks and other machine learning approaches accurately predict patterns in image and time-based data when large historical datasets with curated datasets, excessive compute power including GPUs, and large amounts of time are available. For most applications these prerequisites are not given, and it is important to achieve good predictive results for models that are quickly adapted from a standard pattern to an individual personalized pattern.

IBM Research Zurich has an opening for a student project (preferably Master Thesis, but also Semester Project can be considered) with the objective of achieving an usable workflow for neural network personalization to be used for monitoring activity of daily living of elderly and to evaluate psychological and physiological stress in firemen by using leading hardware (HW) accelerators.

We propose three different HW platforms: Cloud field programmable gate arrays (FPGA https://www.zurich.ibm.com/cci/cloudFPGA/), tensor processing units (TPU), and PULP-based platforms (https://pulp-platform.org). The FPGA accelerator service is accessed via an application program interface (API) and a scheduler. The front-end API receives accelerator service calls through the OpenStack dashboard or a command-line interface. The Coral Dev Board, a compact board with an edge tensor processing unit (TPU) AI accelerator chip speeds up machine learning. A new family of classification models -- Net-EdgeTPU identifies relationships among a baseline AI model's scaling dimensions under a fixed resource constraint (https://coral.withgoogle.com/docs/edgetpu/benchmarks/). The appropriate scaling coefficients for each dimension are then applied to scale up the model to the desired size or computational budget. The PULP platform is an open hardware multi-core platform achieving leading-edge energy-efficiency and featuring widely-tunable performance enabling battery-operated artificial intelligence (AI) in Internet of Things (IoT) applications. It is an open source platform and it comes with an SDK available on GitHub (https://github.com/pulp-platform/pulp-sdk).

The project (Master Thesis) will take place at the IBM Research, Zurich.

Application Scenarios

One application is stress-detection from smart-device data, for activities in extreme environments, (https://researcher.watson.ibm.com/researcher/view_group.php?id=10009). Stress is a root cause for many modern, chronic diseases, thus wearble stress monitoring has a huge potential for stress prevention and management and to improve the quality of life. However, mental stress also critically affects decision making skills. Thus, tools to detect mental stress early, can significantly contribute to work safety in extreme conditions. Traditional stress detection methods are not practical for field-deployment. However, with the availability of low-cost consumer wearable devices that monitor vital signs in real time, more practical stress detection schemes have become possible. Previously, IBM Research measured heart rate variability (HRV) with wearable devices in realistic training environments for firefighters, who were subject to physical, psychological and combined stress. Using machine learning algorithms, different stress types were identified with 88% accuracy, in 1-minute time windows. These predictions are used to help firefighters train more efficiently and experience personal limits, help coordinators to put together the right team for specific missions and finally help mission commanders keep their teams safe. If combined with AI at the edge, to additionally extract context information, real-time closed loop risk mitigation schemes can be implemented with a few seconds latencies. The method is not limited to stress monitoring but can be extended to activity monitoring of daily living of elderly (https://www.activageproject.eu/deploymentsites/Region-Emilia-Romagna/). While activity of daily living personalization and analysis is not as time critical, future expansions, for example fall detection requires faster response times profit from edge acceleration.

Goal & Tasks

Building on the existing model and data provided by IBM, in this project you will:

  • Literature study of relevant background context
  • Achieve a usable workflow for neural network personalization
  • Derive new algorithms to improve performance, also in view of running them on performance constraint embedded platforms
  • Help complete the current dataset acquisition for quantitative stress assessment with additional vital signs like motion or audio and carry out the initial data-analysis
  • Data curation
  • With the new data, assess applicability and portability of personalized deep learning models to generic “community” models and their dependence on personal baselines and expand the model towards federate learning schemes

Prerequisites

(not all need to be met by the single candidate)

  • Knowdleg of Python and some cloud computing
  • Knowledge of machine learning and signal processing
  • Familiarity with deep learning frameworks (Keras, TensorFlow)
  • Motivation to build and test a real system and acquiring field data


Detailed Task Description

A detailed task description will be worked out right before the project, taking the student's interests and capabilities into account.


Status: Available

  • Looking for Semester and Master Project Students
Supervisors: Michele Magno, Xiaying Wang
IBM contacts: Dr. Bruno Michel, Dr. Jonas Weiss


Character

40% Theory and Algorithms
40% Implementation
20% Verification and Testing

IIS Professor

Luca Benini

↑ top

Practical Details