Real-time automated traffic safety monitoring

In many road tunnels worldwide, human operators monitor the flow of traffic through closed-circuit television cameras from a central control room. As they recognize safety risks, such as pedestrians walking in the tunnel or cars stopping, the operators will take measures to reduce the danger, for example, changing traffic signs to a lower speed limit and dispatching safety crew. However, scrutinizing tens or even hundreds of camera feeds over extended periods of time is a cognitive challenge that quickly exhausts humans, who may miss or respond late to a safety incident.

Therefore, Automatic Incident Detection (AID) systems have been developed that help by analysing multiple video streams in real time and highlighting relevant situations. An AID system by ISSD is already in production use in several tunnels in Turkey. Like others on the market, it utilizes traditional computer vision image processing algorithms.

However, these are subject to inherent performance limitations that computer vision approaches based on artificial neural networks can overcome. After encouraging first tests, ISSD were looking to integrate deep-learning technology into their product – but wondered how it would hold up in a full-scale live production environment, and whether the increased computational requirements would still allow them to offer a product at a competitive price point.

Cost-efficient and robust deep learning

In this experiment, ISSD has created a deep-learning based detection engine for their AID product that generates real-time alerts when pedestrians or stationary vehicles appear in a video stream. Together with FED4SAE partners, they closely examined the relevant neural network technology, looking into what too often remains a black box, to better understand its performance and reliability.

Intel’s Movidius™ platform, one of the industrial platforms featured by FED4SAE, proved itself as a capable tool to meet ISSD’s cost target for the server running the detection engine – even at the higher processing load that comes with the neural network based approach.

A key feature of ISSD’s AID system is that it works with a variety of cameras as long as they are network-enabled, allowing it to integrate with a larger range of greenfield and brownfield infrastructure. The FED4SAE platform and advanced technology also fully supported this requirement.

Ahead of the state of the art

ISSD performed detailed measurements of the new detection engine’s performance. To develop a baseline, video footage containing a known number of pedestrians or stationary vehicles was collected from live sites and the current AID system was independently benchmarked against this set. The AI solution was then tested against this baseline, and was found to outperform the current AID solution.

After this, the new detection engine was integrated into a live installation for evaluation at one of ISSD’s customers in Turkey, running in parallel to the proven infrastructure for safety reasons. In this test, the false alarm rate was reduced about half from the legacy solution, with the new engine still detecting all the events that the legacy engine detected, and doing so faster.

The deployment environment for the detection engine employs Intel Movidius Myriad X visual processing units (VPU) in a highly scaled high-density deep learning (HDDL) configuration. ISSD’s configuration pushes the number of Myriad X VPUs in a single server to 56, hosted on 7 HDDL PCIe boards. The specialized VPU hardware performs neural network inference much more efficiently than a CPU, which enables ISSD to process camera video at an even lower hardware cost per stream than with their previous detection engine – even as the processing has grown more complex.

For a look inside the “mechanics” of deep learning that goes much deeper than usual, ISSD applied the open-source Neural Network Dependability Kit (NNDK), another advanced technology promoted by FED4SAE, allowing a better understanding of how the neural network reaches its decisions and how robustly it does so. Among other things, ISSD tested and improved the robustness of the neural network model against image noise that could be introduced by exceptional weather conditions near the tunnel entrance, less-than-perfectly maintained camera lenses or camera electronics.

The new event detection engine significantly strengthens ISSD’s product offering in tunnel safety solutions, with ISSD also planning to apply the new technology to other traffic monitoring products in their portfolio.

FED4SAE support and opportunity

All FED4SAE partners contributed their specific expertise related to technology and product development. Together, the requirements were analysed carefully in order to be able to provide targeted support.

Intel benchmarked the solution across multiple hardware platforms (CPU, FPGA, VPU) to confirm ISSD’s selection, supported with knowledge on how to best use the platform, performance tuning of the neural network and assistance on the server configuration using the HDDL boards.

Leading the technical coaching on neural networks, their architecture and optimization, fortiss supported ISSD in applying the NNDK and interpreting its analysis results to evaluate the object detection network model, and improve its robustness against various kinds of image noise. Based on the analysis results, fortiss also highlighted possible modifications to the model that could significantly reduce its size and speed up inference further.

Blumorpho provided ISSD with focused coaching regarding the business innovation and go-to-market strategy and pitch training. At the Smart Anything Everywhere IoT European Showcase at Digital Catapult London, ISSD had the opportunity to pitch their offering to investors, customers, the academic community and broader industry.
Intel will partner with ISSD (see links below) to assist them in scaling their portfolio of intelligent traffic management solutions to the global market, improving traffic safety worldwide.