Autonomous Driving: Car of the future

Bryce Johnstone from Imagination Technologies discusses what future cars will need from GPUs
Whether it’s new powertrains, in-car infotainment or self-driving cars, automotive technology is advancing at an unprecedented rate. New technologies and industry entrants are challenging many of the fundamentals of what cars are, and, as a result, the driving experience of tomorrow will be very different to what is seen today.
It is expected that by 2030, one in four cars on the road will be self-driving to a degree – in some bounded communities we’re already seeing self-driving vehicles today. But what will happen to the car cockpit? How will the car and driver communicate in the future? As drivers are freed from the responsibility of driving, how will this impact cockpit and infotainment systems design?
Challenges
The car of the future will be very different by the time it becomes fully autonomous around 2025. Some believe it will offer a variety of functions to the occupants during their journey: it will become an office, a living room, a place to rest and a general entertainment centre. It will transform from today’s layout to a more radical design with no steering wheel and seats facing the interior of the car. Without a need for the driver to see everything happening around the car, windscreens and windows will tint, reduce or disappear and cars will become more private spaces. While this will take some time, some of the main trends are already emerging.
It is likely cars will have a multitude of screens, greatly increasing the need for powerful and power-efficient GPUs that can drive a very large number of pixels as well as supporting new features such as augmented reality (AR), gesture control and advanced human machine interfaces (HMIs).
It has been suggested that there could be up to ten screens in next-generation cars, offering a combined resolutions of up to 72k via multiple 4k resolution screens. This would include head-up displays (HUDs) that are starting to be deployed in high-end cars today. HUDs are normally projected onto either the windshield or a specific screen giving an infinite depth, so the driver sees the information in his or her view without having to refocus, as someone would on today’s traditional dashboard.
These HUDs will become much richer and more complex with a windshield-based split-screen HUD with passenger and driver portions that can only be viewed by the person sitting in the driver or passenger seat. At the core of this is GPU technology, which will not only be pushing pixels to the screen on the dash but will be placing them on the HUD unit too.
Technologies such as gaze tracking will be coupled with the HUD in cars that have drivers to place information relevant to driving at the centre of the driver’s line of vision. Gaze direction can also be used to determine whether the driver is paying attention to the road or not and algorithm tuning on the GPU can make the calculations to warn the driver if they are not fully concentrating and should pull over.
HMI
As cars progress from level one to level five autonomous driving, the HMI will become increasingly important in levels two and three driving where control needs to be passed back to the driver. Also, the driver will need to be notified about actions the car is taking on behalf of the driver so they will not be spooked by unexpected behaviour.
Today, interaction with the dashboard is largely visual. Increasingly this will be enhanced by voice interaction, audio responses, vibration alerts, gesture controls and projected visual warnings onto an HUD.
New HMIs will play an important social role by helping users adjust to the new reality of autonomous cars. During the transition, people will have to learn to trust driverless vehicles. A passenger should at any time be aware of what is happening in the car: why it chooses the specific lane, which cars are around, how busy one or another road is and how the route is being calculated. A well-designed HMI, incorporating graphics and audio elements, will be fundamental to the acceptance of autonomous vehicles.
The HMI should offer a natural way to visualise the decision processes in the car so the passenger feels safer and more comfortable. For example, automotive AR could use a combiner glass with a holographic film on it, which works as a lens and reflects only certain wavelengths. People will look through the combiner glass in front of the windshield and see the projected video or interactive interface.
Of course, difficulties will emerge. For example, there will have to be a change to one of the essential drivers’ habits of looking in the mirrors before manoeuvring. Also, developers still have to prove that the new elements of HMI provide a better and more precise experience than today.
AR
So what will AR bring to the car in the next decade? AR is a technology where virtual world objects are placed over the real world to enhance or augment what drivers are viewing with icons, objects, information and so on.
Some companies have posited the use of the front screen as a projection for the AR HUD. Smaller unobtrusive versions of this exist but future ones have the potential to be all encompassing. The AR view would be split across the windscreen into a simplified driver view; this would reduce distractions and only deliver key vital information to the driver.
On the passenger side, much richer content could be displayed, such as the location of restaurants, shops, the nearest parking space and other points of interest. One of the other advantages of an AR HUD is that the gaze direction can be inferred by in-car cameras and the key driving information projected into the centre of the gaze to allow for an expected amount of head movement.
GPU compute
As the industry moves towards fully autonomous level-five driving, more driver assist functions are required in the car. These include the likes of emergency automatic braking, lane departure warning, pedestrian detect, driver attention, blind spot detect and cross traffic assist.
Advanced GPUs are parallel processing units and can be used for repetitive algorithms such as those used in many of the functions for adas. The GPU’s parallelism and multiply and accumulate structure is perfect for implementing the likes of convolutional neural networks (CNNs).
CNN algorithms have been around for over 30 years and until recently they were the preserve of server farm processing. However, with advances in SoC process technology as well as improvements in CNN algorithm efficiency and GPU advances, these can now begin to be implemented at the edge of a network and not just in the cloud.
Looking at adas functions, anything that requires some level of image processing, such as extracting road-sign information, is well suited to using CNNs to improve overall performance. In comparison to a standard high-end CPU, an embedded high-end GPU can offer over twenty times the performance at a much lower power dissipation.
Ultimately, CNNs will be accelerated in hardware as their function become well understood and perhaps standardised. However, GPU compute has a place to enable newer types of neural network and other acceleration to be prototyped and deployed prior to full optimisation.
To improve safety, hardware virtualisation can be leveraged by the various applications and services being deployed to the car. GPUs can be hardware virtualised so that no two OS and application combinations can use the same memory space. The cores can have a two-level MMU to ensure absolute compliance to this requirement. In the case of a rogue piece of software being downloaded, only that container of an OS or application running on the secure hypervisor will crash. Nothing else will be affected, so, for example, the infotainment system might crash but, crucially, the adas and cluster and HUD elements running on the GPU won’t be affected.
Future
GPU companies have to look at autonomous driving well in advance and deliver silicon IP four to five years in advance of it appearing in a car. So what is designed into the car in 2018 will be in the 2023 model. What will be delivered in the 2021 timeframe is already largely designed into silicon today.
The world is entering one of the most exciting periods of revolution in automotive design. Key to this are the changes to cockpit designs that will materialise as we move to increasing levels of autonomous driving. To deliver the required feature richness and adas functions within an acceptable power budget, the HMI and its various elements will rely heavily on the processing capabilities of tomorrow’s high-performance and low-power GPUs.
Bryce Johnstone is automotive segment director for Imagination Technologies
