Infotainment: We are not alone

Jatinder Singh from Lattice Semiconductor looks at how FPGAs can enable mobile interface bridging in adas and infotainment applications
In the automotive market, as in all industries, competition breeds innovation. Over the past 100 years, this truth has transformed the horseless carriage into one of the most useful and ubiquitous tools in the modern world.
Automobiles have long been understood as mechanical devices but, recently, continuing innovation has transformed them into increasingly electronic systems.
One obvious example of that is the rapid electrification of the car. But equally significant has been the evolution of entertainment from simple radio players and tape decks to powerful automotive infotainment systems (AIS), and more recently the introduction of advanced driver assistance systems (adas) that help protect drivers with enhanced electronic safety systems.
Over the past decade, one of the primary drivers of innovation has not come from inside the automotive market, but from the consumer electronics market. The rapid rise of the smartphone has forced car makers to adapt quickly to a new device that has quickly become ubiquitous.
Since the 1950s, the automotive entertainment system has been primarily based on radio, tapes or CDs. While some navigation and other so called infotainment systems had already been introduced by the time the iPhone was unveiled in 2007, their functionality was limited and very expensive.
The introduction of the smartphone upended this paradigm, introducing a platform and ecosystem of applications into the car that the car manufacturer had no control over. Key apps, such as Google Maps that could use the GPS built into the phone to provide turn-by-turn directions had a transformative effect on the driver’s experience. Entertainment changed too. Instead of being forced to listen to the radio, or whatever CDs they had lying around in the car, drivers could now access millions of songs on custom playlists.
The challenge from smartphones accelerated trends that were already taking place in the automotive space and in response the entertainment system in a car increasingly became an AIS, with a goal not only to entertain, but to inform.
At the same time, automobile manufactures had been developing new ways to protect their occupants. Adas includes a range of products such as automatic braking, lane detection and birds-eye parallel parking that use advanced technology and processing to enhance driver safety.
To power both their AIS and adas equipment, automotive manufacturers have looked to the mobile space. The processors and systems powering smartphones are a good candidate to drive these systems, sharing similar constraints in size, power consumption and wider temperature range. In addition, the volumes in which they were produced ensured that the car manufacturers had an inexpensive platform from which to design.
However, there are challenges on this path as well. Cars and smartphones have vastly different design cycles – nine to18 months versus three or more years. Because of this a smartphone processor platform can be at the end of life before the automotive system designed around it could even reach the market.
Video is also an issue. Smartphones generally have two inputs for video to support the front and back camera. An AIS or adas by contrast could be receiving and analysing video from four or more cameras, or other sensors.
The desire to use more powerful processors to power advanced automotive systems looks set to continue, as more cameras and more screens appear inside the car over the next decade. Given the benefits, it seems likely that the car manufacturer will continue to try to adapt existing mobile processors to meet this need. Because of this, there is a need for flexibility to help adapt the mobile processor to be an application processor for the automotive market.
FPGAs are particularly well suited for these sorts of bridging applications. Modern FPGAs can provide a needed degree of flexibility, while still maintaining an efficient cost, low power consumption and powerful features. In particular, FPGAs can help resolve the mismatch in the video inputs, outputs and connectivity in the modern automobile, reducing cost and time to market.
Various FPGAs and ASSPs are available in an automotive grade, including AEC-Q100 certification. These will be increasingly important, as the AIS of today becomes the foundation of the adas driving the automated car of the future.
Infotainment
When the Tesla Model S was introduced in 2012 one of its many innovative features was the large 43.5cm infotainment touchscreen in the centre console. Also notable was the lack of physical controls. Outside of the core driving components, everything else, hvac, entertainment and information had all been pushed to this one giant touch display.
While the Tesla console screen is remarkable for its all-encompassing size, it is a natural evolution of display proliferation inside the car cabin. Video displays first showed up in the centre console in the entertainment system of the car. From there they expanded to the headrests to provide rear-seat entertainment. Today they have expanded further to digital instrument clusters and head-up displays.
Like displays, cameras first showed up in the car for a single purpose, to help drivers see what was behind them as they were backing up. Since then their role has expanded. In today’s car, cameras can be used to analyse a car’s surroundings and information overlaid the augmented reality through a head-up display. In some models of cars, the side mirrors are beginning to be removed in favour of cameras that are tucked into the car’s body, reducing wind resistance and improving fuel efficiency.
As the number of cameras and screens connected to the AIS has proliferated, it has become harder to adapt the mobile processor platforms to these systems. Most smartphones have two cameras, a primary rear-facing camera and a front-facing camera for videoconferencing. But when applied to the car space, with its multiple cameras, the two inputs available are insufficient.
A second challenge is dealing with the video interfaces. Mobile phone processors generally have a single DSI output for display, however screens in the automotive space largely use LVDS, which many mobile processors do not support.
FPGAs help automotive infotainment systems adapt to these challenges by pre-processing video signals for customer resolutions, bridging various interfaces throughout the car, and functioning as a serialiser-deserialiser (serdes) allowing multiple video screens to be driven by a single video output.
Adas
Adas refers to various new technologies that help the driver on the road. These systems use an array of advanced sensors that inform the driver to dangers up ahead, and in some cases automatically compensate for changing road conditions.
While today assistance is as far as the mass market systems go, already companies from Tesla to Google are testing semi-autonomous and fully autonomous systems that one day soon will bring passengers safely from a starting point to their destination with minimal input from the driver.
Adas depends on the sensors that help it take in data about the world around it. These can include cameras, radar, lidar and others. Whereas in older cars, a camera was primarily used to assist in backing up, in an adas the video from an array of cameras is seamlessly transformed into a bird’s eye view of a car and its surroundings. Cameras can also be used with radar sensors for parking assistance and automatic braking or for drive recording to a black box that allows insurance agents to investigate the aftermath of an accident.
Many of these systems require data from multiple cameras and other sensors to be seamlessly aggregated for the system to properly analyse its surroundings.
To analyse a car’s surroundings more accurately, adas cameras will become higher resolution, with higher frame rates and higher colour depths. This will allow systems to analyse more data, but also require significantly more bandwidth to be supported. Whereas a basic rear-view camera system still relies on low-resolution analogue connections, adas cameras require higher speed digital interfaces, such as CSI-2.
Here, as in infotainment, the limited IOs of a mobile processor mean bridging is required to adapt it to the needs of automotive. With FPGAs, multiple camera and other sensor data will need to be aggregated through a single camera input on the processor. More powerful FPGAs can help pre-process data and even control sensors as well.
Conclusion
The flexibility of FPGAs has already proven invaluable in many industries. In automotive, FPGAs can help bridge the gap between the needs of advanced entertainment and safety systems, and the mobile processors on which they run. This business model has distinct advantages, allowing automotive manufacturers to use mass market proven products from the smartphone space and rapidly adapting them to the changing needs of the car. FPGAs are also now beginning to show up in areas such as motor control in other industries and could prove useful in automotive as well.
One thing is clear, as car electronics continue to advance, and especially as adas and other efforts continue to move the industry towards the driverless car future, more and more sensors will need to be integrated into these systems, and the need for FPGAs to provide the flexibility to adapt cameras, sensors, video and higher speed connectivity to the changing needs of the market will only grow.
From a Lattice Semiconductor white paper by Jatinder Singh, product marketing manager of the company’s industrial and automotive division
