Infotainment: No need to touch

Haptic technology can provide tactile feedback for touchless gesture control

David Owen from Ultrahaptics discusses next-generation interfaces in automotive infotainment systems, looking to a future of touchless gesture control with haptic feedback

The pervasion of electronic functionality in cars is perhaps most readily visible in the automotive infotainment area. Leading car makers are building increasingly complex infotainment systems, typically offering a proliferation of functionality and customisable settings.

This trend clearly requires the development of a more sophisticated user interface, but no system can make too many compromises on ease of use or it will risk user acceptance.

Today, many different user interface options to control in-car infotainment systems are being integrated into the car, such as steering-wheel controls, multi-function buttons and knobs, as well as voice recognition.

A major trend has been the integration of increasingly large displays for in-car infotainment. In concert with this trend, new car models are likely to offer the replacement of traditional buttons, switches and dials with digital display touchscreen controls. Automotive makers have recognised that users are increasingly seeking automotive interiors and dashboards that have a more modern look, yet also offer an easy-to-use interface.

Use of fingers to make touch-based gestures is now commonplace when using smart consumer devices, for example. However, touch-based gesturing has some clear limitations for use in the car: pinching and scrolling on a display while driving clearly brings safety risks with driver attention being diverted from the road.

 

Touchless

An increasingly interesting option for both users and the industry is touchless gesture recognition based control, which can provide the next step for automotive infotainment. The drive for touchless gestures comes with the use of gesture-tracking technologies such as those that employ high-definition cameras. These can be programmed to detect hand motions and to disregard other movements.

The range of potential touchless gestures available to drivers will clearly depend on each system, but touch-based user interfaces employed in mobile devices have shown that common gestures can be quickly and easily learnt.

Unfortunately, the car industry has found a major issue with gesture-based interfaces. First, a driver might not immediately know if their hand is even in the right area for the gesture to be recognised; and, secondly, there is often a time delay between gesture and action and the driver may not be aware that the gesture has been registered and the system is simply processing the command. Users often find they need to repeat the gesture to make sure the device understands it, potentially cancelling the desired command, or they need to take their eyes off the road and check the display visually, as with touch-based control.

 

Feedback

A highly attractive solution to these problems is haptic feedback, which is the sensation of touch and is already well known in computer gaming and many mobile consumer products. It has been achieved so far through comparatively simple implementations that have become practically synonymous with touch technology. Haptic feedback has also started to appear in the automotive arena, primarily in adas.

For example, some systems will vibrate the steering wheel when the car begins to stray from its lane on the road or use actuators in the seat to simulate the effect of driving over the rumble strips that often lie to the side of the road.

Although tactile feedback provides a more complete user experience, closing the loop to provide helpful non-visual cues, most approaches are limited by the need to be in physical contact with the haptic system, which means they cannot be used in applications based on touchless gesture recognition at a distance.

To deliver a touchless haptic system requires new technology. To address this, an acoustic radiation force can provide mid-air user feedback for touchless gesture recognition. Focused ultrasound techniques can augment recognition and control systems for hand gestures.

 

Ultrasound

The key to the technology’s ability to create haptic feedback lies in the interaction between focused ultrasound and the skin. When the ultrasound is focused onto the surface of the skin, it induces a shear wave in the skin tissue. The displacement caused by the shear wave triggers mechanoreceptors within the skin generating a haptic sensation.

It generates ultrasound using transducers mounted under the fascia of a product or on a separate control panel to provide the sensation of touch up to a meter away from the surface. An array of ultrasound transducers creates one or more focal points of energy, which can be less than 1cm across, on the user’s hand.

Additionally, modulating the transducers’ ultrasound frequency allows different types of sensation to be created.

The combination of these two approaches enables software to generate a wide range of touch-based sensations.

Algorithms can control the volumetric distribution of the ultrasound force field to form various three-dimensional shapes. For example, a close focused beam could simulate the curve of a knob or button, whereas a rapidly modulated beam of energy may simulate the feeling of rubbing the hand across a ribbed surface.

A suitable ultrasound carrier frequency is 40kHz due to availability of production-ready transducers and its propagation and audible properties. However, the receptors within human skin are not able to detect vibration at such a high frequency; they can only detect changes in intensity of the ultrasound. The 40kHz ultrasound therefore needs modulating at a lower frequency within the perceptual range of feeling.

To determine the location of a driver’s hands or fingers, the technology can work with a range of input sensors, such as a gesture-tracking leap motion camera held in a cradle in a pre-calibrated position or potentially with radar-based technology.

The system can also be set up to deliver feedback to confirm that a driver’s hand is within the interaction zone. For example, passing the hand through a wall of acoustic force to indicate to the driver that they have passed though one of the outer boundaries of the interactive zone and are now able to make gestures to control the infotainment system.

Another significant capability is automatic recognition and location of the gesture interaction zone. Essentially using the camera’s image-processing ability, the system clamps itself to the hand – firing the required transducers in the appropriate sequence – and the acoustic-force-produced button, switch or dial will continue to be available to the driver.

 

Summary

The advantages of not having to touch in-car equipment are highly valuable. The integration of haptic feedback into touchless gesture recognition can help car makers gain greater market acceptance for the technology to deliver control of infotainment systems. In the future, this could also be extended to passenger-comfort functions such as the heating and air-conditioning system or automatic windows, sunroof and seat controls.

David Owen is vice president of business development at Ultrahaptics

www.ultrahaptics.com

 

Recommend to Colleagues: