Neural Network Dependability Kit

A toolbox to support safety engineering of artificial neural networks

In recent years, neural networks have been widely adapted in engineering automated driving systems with examples in perception, decision-making, or even end-to-end scenarios. As these systems are safety-critical in nature, problems during operation such as failed identification of pedestrians may contribute to risk behaviours. Importantly, the root cause of these undesired behaviours can be independent of hardware faults or software programming errors but can solely reside in the data-driven engineering process, e.g., due to unexpected results of function extrapolation between correctly classified training data.

The Neural Network Dependability Kit (NN-dependability-kit) is an open-source toolbox to support safety engineering of neural networks. It supports verification, test-case generation and metrics computation for neural networks. The key functionality includes:

• Formal reasoning engine for ensuring that the generalization does not lead to undesired behaviours, and

• Novel dependability metrics for indicating sufficient elimination of uncertainties in the product life cycle

• Runtime monitoring for reasoning whether a decision of a neural network in operation time is supported by prior similarities in the training data.

The NN-dependability-kit is available for download at GitHub, where further information can be found: https://github.com/dependable-ai/nn-dependability-kit

Examples of NN-dependability-kit use cases:

1. Formal Verification of a Highway Front Car Selection Network

Formally verifying properties of a neural network that selects the target vehicle for an adaptive cruise control (ACC) system to follow. The overall pipeline is illustrated in the figure, where two modules use images of a front facing camera of a vehicle to (i) detect other vehicles as bounding boxes and (ii) identify the ego-lane boundaries. Outputs of these two modules are fed into the third module called target vehicle selection, which is as a neural-network based classifier that reports either the index of the bounding box where the target vehicle is located, or a special class for “no target vehicle”.

The input features of the Target vehicle selection neural network are defined as follows: 1-8 (possibly up to 10) are bounding boxes of detected vehicles, E is an empty input slot, i.e., there are less than ten vehicles, and L stands for the ego-lane information.

A tutorial for this example is available at the GitHub repository: https://github.com/dependable-ai/nn-dependability-kit/blob/master/TargetVehicleProcessingNetwork_FormalVerification.ipynb

2. Perturbation Loss over German Traffic Sign Recognition Network

Analysing a neural network trained under the German Traffic Sign Recognition Benchmark with the goal of classifying various traffic signs. With NN-dependability-kit, one can apply the perturbation loss metric, in order to understand the robustness of the network subject to known perturbations.

A tutorial for this example is available at the GitHub repository: https://github.com/dependable-ai/nn-dependability-kit/blob/master/GTSRB_AdditionalMetrics.ipynb

Framework for Distributed
Industrial Automation & Control

Today’s automation and control systems are mostly implemented according to a vendor specific dialect of the IEC 61131 standard. The different dialects make programs for programmable logic controllers (PLC) hardly portable between different PLCs. With its PLC centric design and its scan-based nature, the IEC 61131 standard already exceeds the needs of today’s automation systems. Therefore the same standardization group, originating IEC 61131, worked on a more flexible version, supporting distributed automation and control systems, the IEC 61499 standard.

IEC 61499 based systems follow an application centric design, which means that the application of the overall system is created at first and independent of the hardware. Each application is created by interconnecting the desired function blocks (FB) in terms of a function block network (FBN). The IEC 61499 standard extends the FB concept, already introduced within the IEC 61131 standard by an event interface, which allows an explicit definition of the execution order. This is required especially for distributed systems.

As soon as the hardware structure is known it can be added to a project’s system configuration and the already existing application can be distributed onto the available devices. The interoperability of devices from different vendors is one key feature postulated by the IEC 61499 standard. Also the portability of applications between different IEC 61499 engineering tools has been addressed by introducing an XML exchange format. Due to the definition of management commands different devices can be configured by different IEC 61499 engineering tools. The same commands can be used for online reconfiguration.

Eclipse 4diac™ implements the IEC 61499 standard and is intended for the programming of programmable logic controllers (PLCs) as well as small embedded control devices. 4diac is provided as open source software under EPL-1.0 and consists of two parts: forte (4diac-rte) and 4diac-ide.

• forte is a real-time capable IEC 61499 run-time, which operates on different hardware platforms. The hardware platforms reach from small embedded control devices up to PLCs. Currently supported hardware platforms comprise e.g. Weidmüller PLC, Wago PLC, Raspberry Pi, BeagleBone Black, or LEGO Mindstorms NXT. forte has been tested on e.g. Windows Cygwin on i386, ppc and xScale Linux on i386, ppc and xScale NetOS RTOS on IPC@chip and eCos on ARM7. It supports all IEC 61131-3 edition 2 elementary data-types, structures, and arrays. Applications can consist of any IEC 61499 element as basic function blocks, composite function blocks, service interface function blocks and adapters. Besides that forte provides a flexible and extendable communication infrastructure already providing a large set of protocols, e.g. MQTT, OPC UA, openPOWERLINK or EclipseSCADA.
• 4diac-ide is an extensible, Eclipse-based integrated development environment (IDE) for IEC 61499 compliant programs. It supports the modelling of distributed control software as well as the deployment to the various hardware devices. 4diac-ide makes it easy to create new applications in a modular way by reusing existing function blocks from standard libraries. The modelled applications can be deployed to IEC 61499 powered control devices. For testing, 4diac-ide provides monitoring and debugging facilities to “watch” events and data values during execution. One can also interact with the application by changing or forcing data to have certain values, and by triggering events.

4diac has been evaluated by case studies of different organizations:

• The Profactor Case Study. Profactor is located in Steyr, Austria and is Austrian’s No. 1 in applied production research. As a private research company, it performs applied research across different disciplines to find solutions for the manufacturing industry. The Profactor case study employed the 4diac platform on a robotic arm application.
• The AIT Case Study. The AIT (Austrian Institute of Technology) is Austria’s largest non-university research institute, and among the European research institutes a specialist in the key infrastructure issues of the future. In this case study AIT examined the use of the 4diac platform in a smart grid laboratory.
• The ACIN Case Study. Vienna University of Technology – Automation and Control Institute (ACIN) performs research and education in the fields of distributed automation and control systems, as well as precision engineering, scientific instrumentation and process-measurement systems with focus on industrial relevant applications for industrial automation, production, and measurement systems. The ACIN case study allowed the Vienna Institute of Technology’s Automation and Control Institute to demonstrate the real time capability of the 4diac platform.
• Awite Bioenergie GmbH. Awite is a SME located near Munich and is specialist for gas analysis and desulfurization as well as the automation of the corresponding processes. Within their CPSE Labs Experiment they implemented an energy load management approach with 4diac. They summed up their results within a YouTube video.

For more info please contact

Dr. Holger Pfeifer

Email: pfeifer@fortiss.org

https://www.fortiss.org/home/

Advanced nanotechnology for chemical sensing

From nanotechnology to sensing applications
From nanotechnology to devices

WHAT IS our technology

The miniaturization of sensors together with the dramatic increase in portable computational power is currently generating a wide range of new applications for chemical sensors. These sensors can be used for applications related to environmental, health, aeronautics, or even food safety monitoring just to name a few. In order to be used with portable devices, sensing materials must be small, cost effective, and reliable. Sensors based on luminescence changes in the presence of specific molecules are promising candidates for this type of applications as these sensors can be interrogated with standalone compact optical readers that have wireless communication capabilities. CSEM has developed new optical sensitive patches based on a sol-gel nanoporous layer and these luminescent films are adapted for O₂, CO₂, and pH detection.

Nanotechnology based sensitive layers:

To make the most of the photosensitive dyes, new functionalized thin films based on nanoporous layers have been developed and can be deposited on various substrates such as steel, glass, and flexible plastic sheets. The host film is made of a double matrix: a microporous sol-gel network encapsulating the active dyes, which is embedded in a mesoporous coating. This hierarchical nanostructure brings enhancement to the sensing layers properties, such as optical signal, sensitivity, robustness, mechanical resistance, transparency, selectivity, and response time.

These nanoparticle based layers can also be deposited on MEMS structures to build sensors (electrical, electrochemical) using the nanoporous layer as a catalyst for sensing materials.

APPLICATIONS

Miniaturization of sensors for different connected applications

• Oxygen sensing in pressure sensitive paint (PSP)
• Oxygen sensing in cell culture, air quality, water quality, and breath monitoring
• Carbon dioxide sensing in buildings, food packaging, food preservation, and breath monitoring
• Carbon dioxide, oxygen, and pH sensing for water quality (rivers, swimming pools)
• VOC for buildings, cars

What’s new?

• Patented technology (EP 3184994, US 2017176332)
• Deposition of the sensing layer with high resolution on structured or fragile substrates (membranes)
• Reversible and disposable sensors for continuous monitoring
• O2 sensing capabilities with accuracy <0.2%, range 0-21%, and precision 0.3%
• CO2 sensing capabilities with accuracy 0.2%, range 3-12%, and precision <1%

WHAT’s NEXT

• Customization of sensor integration for extending the range of application
• Wearable sensors for health monitoring
• Adaptation of the sensing layer to new chemicals

INTERNATIONAL RECOGNITION:

• Innovation award CCI FS 2016 (https://www.csem.ch/Page.aspx?pid=37881)
• BioInnovation-Eclosion Prize 2015 (https://www.csem.ch/Page.aspx?pid=32622)

FOR MORE INFORMATION

Contact: Guy Voirin
Email: guy.voirin@csem.ch
Tel: +41 032 720 5152

Localization Solver

A GPS Free Generic Radio Localization Solver.

WHAT IS A GPS FREE LOCALIZATION SOLVER?

The aim of the positioning solver is to allow localization of any communicating device in a reliable manner whether indoor or outdoor. Based on the information collected directly from the radio, the solver is capable of parsing, filtering and analysing the transmission quality to integrate a reliable position.

Tailored around a particle filtering technique, information collected from the radio environment is transformed into a series of possible points, called particles. For each of these particles, a probability of being the searched position is calculated. The probabilities of all particles are processed through an iterative process until convergence.

CSEM GPS FREE LOCALIZATION ALLOWS:

• Localization of any objects communicating wirelessly provided infrastructure.
• Integration with any type of radio capable of generating RSSI, SNR, ToF, AoA, TDoA, DTDoA (LoRa® / LTE-M / NB-IoT / WiFi / BT or customised hardware)
• Tracking devices without modifying the infrastructure nor the hardware allowing low power localisation
• Easy port to handheld device or cloud servers (AWS, Microsoft Azure, …)
• Early data fusion to integrate obvious data rejection (Map Matching …)

APPLICATIONS

• Logistics, Security
• Pets / Objects / People Tracking / Finding
• Occupancy detection
• Drone navigation and obstacle avoidance
• Planification

What’s new?

• By using random based initial distribution, the caveats of the generally used geometrical calculation is greatly reduced (less impact of local minima)
• Solver was intensively tested on LoRa® network and RSSI with results 10 to 50% better than evaluated competition

WHAT’s NEXT

CSEM will continue to develop its GPS Free Localization Solver to improve its level of precision. The next path to be evaluated is the usage of Machine Learning to discriminate bad measurements and limit the impact of multipath.

Depending on the need of our partners, we are introducing probability based constraints to early adapt to other types of information like proximity, room, path …

FOR MORE INFORMATION

Contact: Martin Sénéclauze
Email: martin.seneclauze@csem.ch
Tel: +41 32 720 53 40