Neural Networks: Way forward

Imagination Technologies

Bryce Johnstone from Imagination Technologies believes that convolutional neural networks will be essential for autonomous driving

It’s no surprise that neural networks are becoming increasingly prevalent in a wider variety of markets. They tailor people’s social media feed to match their interests, enhance their photos to make them look better, and power the feature detection and eye tracking in AR and VR headsets.

They can be used for security, offering improved facial recognition and crowd behaviour analysis in smart surveillance and are smarter than humans at detecting fraud in online payments systems.

When it comes to automotive, neural networks are going to be the power behind the driverless car systems for collision avoidance and subject tracking. Applications for neural networks in vehicles include driver alertness monitoring, driver gaze tracking, seat occupancy, road-sign detection, drivable path analysis, road user detection and driver recognition. As the number of autonomous vehicles and smart transportation systems increases over the next few years, these applications will also continue to expand. 

 

Beyond adas

As we move through the levels of autonomous driving, the need for more sensors and specifically cameras is paramount. The vehicle needs to understand and interpret its surrounding through lidar, radar and cameras, as well as infra-red and other technologies. Sensor fusion is the point at which the information extracted from each of these sensor types is merged together and an understanding of what is happening in the car’s immediate environment becomes understood so the vehicle or driver can react.

Moving from level two, where the driver's hands are still on the steering wheel and the driver is still engaged, up to levels four and five means that the processing has to handle a lot more data from more sensors. It has to process these speedily enough so appropriate actions can be taken, such as automatic braking and automatic steering for example.

 

Hardware

To address the challenge of extracting useful information from the multiple camera inputs, designers leverage the inherent efficiency of convolutional neural networks (CNNs).

CNNs are a form of machine learning that process data in a similar way to the human brain. Neural networks use different layers of mathematical processing to make, over time, increasing sense of the information they receive, from images to speech to text and beyond. For neural networks actually to work, they must be trained by feeding them massive amounts of data.

With a CNN, this might be done for image classification. In the case of a road-sign recognition system, hundreds of thousands, if not millions of images of road signs at different angles, distances, exposure levels, colours and shapes would need to be run through the system. A set of coefficients for the various nodes and layers of the CNN would be built up so the particular road sign could be identified.

There is a stage called inference, which as it suggests infers new data coming into the sensor and decides whether this is a road sign or not. The neural network learns by training offline initially and then has the possibility to learn more in real time through inference. Training a network for high accuracy takes a large amount of compute power and is often done on large arrays of GPUs in data centres.

CNNs can run on a generic computing resource such as a CPU or a GPU using its compute capabilities. GPUs are inherently better than CPUs at performing neural network algorithms providing a huge boost in performance as well as a large reduction in power dissipation. However, if dedicated hardware blocks optimised uniquely for the types of networks and algorithms being run are targeted, there is an opportunity for yet another step-change in performance as well as a reduction in power. By driving the vectors of power and performance, highly complex neural network calculations can be executed at the network edge without requiring the work to be done in the cloud.

This addition of dedicated silicon for specific use cases is a natural evolution of hardware. Originally, the early desktop processors didn’t even have a maths co-processor for accelerating floating-point calculations on which applications such as games are so dependent, but since the 1980s it has been a standard part of the CPU.

In the 1990s, CPUs gained their own on-board memory cache to boost performance, while still later GPUs were integrated. This was followed in the 2010s by ISPs and hardware support for codecs for smooth video playback. Now it’s the turn of neural networks to get their own optimised silicon and be a standard part of computing devices on the edge of the network.

 

Keeping it local

In many cases, the inferencing could be run on powerful hardware in the cloud but, for many reasons, it’s now time to move this to edge devices. Where a fast response is required, it’s simply not practical to run neural networks over the mobile network due to latency issues and, worse, connection issues.

Moving it on-device also eliminates many of the potential security issues that are becoming a bane of the automotive industry. As cellular networks may not always be available, be they 3G, 4G or 5G, dedicated local hardware will be more reliable, offer greater performance and, crucially, much-reduced power consumption

A car typically drives at speeds of 110km/hr (34m/s) on British motorways. Without hardware, it would need to anticipate obstacles 20 to 50m ahead to avoid a collision but because of latency, bandwidth and network availability, it’s impossible to do this over the cloud.

With a true hardware system, the car can run multiple neural networks to identify and track objects simultaneously at only 1m distance.

These days, smartphones are repositories of pictures, and typically there might be 1000 or more photos on the device sorted automatically in various ways, including, for example, identifying all photos featuring a particular person. This requires analysis; a premium GPU running a neural net could do the process in around 60 seconds, but a dedicated CNN accelerator can do it in just two seconds.

Then there’s battery life. A GPU could process around 2400 pictures using one per cent of the battery power. Conversely, consuming the same amount of power, a neural network accelerator can handle 428,000 images.

This low power consumption can enable new use cases, for example, in areas such as smart surveillance. Such an accelerator can be powerful enough to perform on-device analytics, whether in a camera in a city centre, a stadium or a home security system. Due to its ability to run multiple different network types, it can enable more intelligent decision making, thereby reducing false positives. Thanks to the low-power processing, these cameras can now be battery-powered, making them easier to deploy and manage.

 

Flexible support

To make these use cases possible, neural network accelerators have been designed from the ground up for efficient neural network inferencing. So what makes hardware acceleration technology different from other neural network options, such as DSPs and GPUs?

First, there are the ultra-low power consumption characteristics that leverage expertise in designing for mobile platforms. The second factor is the flexible bit-depth support, available crucially on a per-layer basis.

A neural network is commonly trained at full 32bit precision, but doing so for interference would require a very large amount of bandwidth and consume a lot of power, which would be prohibitive within mobile power envelopes, that is even if the performance was available to run it, the battery life would take a huge hit.

To deal with this, certain advanced accelerators offer a variation of the bit-depth for both weights and data, making it possible to maintain high inference accuracy, while drastically reducing bandwidth requirements, with a resultant dramatic reduction in power requirements.

A brute-force approach is not needed to reduce bit depth. It can be varied on a per-layer basis for weights and data, so developers can fully optimise the performance of their networks. Furthermore, it maintains precision internally to preserve accuracy. The result is higher performance at lower bandwidth and power.

In practice, such implementations require as little as a quarter of the bandwidth compared with other options. Moving from 8bit down to 4bit precision for those use cases where it is appropriate enables devices to consume 69% of the power with less than a one per cent drop in accuracy.

 

Conclusion

As the world becomes ever more expectant of computers and devices having a greater understanding of the world around them, neural network acceleration represents the most efficient way to extract meaningful information out of the camera captured image. This advancement is sitting at the heart of autonomous driving as the world moves to partially and fully autonomous driving over the next decade. The need to be able to process these heavy algorithms in real time and within ever stricter power budgets means that using edge devices will be crucial to the success of this autonomous vehicle technology.

Bryce Johnstone is director of automotive at Imagination Technologies

www.imgtec.com

Recommend to Colleagues: