04 Feb Smart Surfaces, Sensors, and the Future of Automotive HMI
By Ali Foughi, NextInput CEO and Co-Founder | Tom Gilbert, Executive Director Business Operations 10.13.2020
We are truly in the golden age of the automobile. Technological innovations over the past fifteen years have created the modern vehicles we see on the road today. These innovations have been key to improved reliability and safety, reducing the death rate by approximately 25% per million miles driven. And yet, despite these advances, distracted driving is the primary contributing factor in up to 58% of automobile accidents.
Rethinking the Automotive Cabin
Beginning in the 1980s and 1990s, the automotive cabin transformed from the utilitarian design of the 1950s into something more akin to an airplane cockpit. Gone were the days where the only electronic device available to the driver was the radio. Electronic door locks, power windows & mirrors, CD changers, air-conditioning, cruise control, variable-timed wind-shield wipers, and even heated seat controls became standard, each getting their own set of buttons. Jump to today with Bluetooth pairing, mapping software, and large touch displays with full Internet connectivity, functionality has grown beyond our ability to navigate all the information available to us while remaining concentrated on the road ahead. The automotive industry is being forced to rethink their Human-Machine-Interface (“HMI”) in order to satisfy consumers’ demand for ever-increasing content and utility, while re-engineering to a slick, minimalist design that is both aesthetically pleasing and safe to navigate.
Mercedes Benz VISION EQS Conceptual Smart Surface Interior © Daimler AG
The Smart Surface
Automotive manufacturers first took their cues from the modern smartphone; creating touch displays with proprietary Operating Systems with various controls compartmentalized into menus and sub-menus. These early adoptions failed for two main reasons: First, the menus were difficult to navigate. Unlike a smartphone that has your undivided attention, the lack of a physical button with intent-driven tactile response frustrated drivers trying to multi-task. Second, existing display technology, while good enough for smartphones, was inadequate for the automobile. Resistive touch displays were difficult to read in direct sunlight and performance significantly degraded over the years. Likewise, drivers found capacitive touch unaccommodating as well. In cold weather, drivers could not operate the HVAC without taking their gloves off. Environmental conditions such as moisture and EMI interfered with the system’s ability to accurately recognize the user’s input, significantly degrading the user experience. Technology in the automobile must remain responsive in any environment to prevent distracted driving and avoid hazards.
In response, industrial designers created concept vehicles and renderings that reimagined the dashboard from having stand-alone buttons to surfaces that had the same functionality, except hidden behind luxury surfaces: buttons available upon demand that would disappear again when no longer needed. They imagined smooth, gapless, solid wood or carbon fiber panels that would just show the controls relevant to the driver. Not only would these new interiors be beautiful, but they would be practical as well. By removing traditional buttons, they would also eliminate the number one warranty expense in the modern automobile: mechanical button failure. Under even ideal conditions, a typical mechanical switch will fail after a few hundred thousand actuations. Add in ingress from oil, dirt, and abuse, and the time to failure shortens. Strain gauges were used in concept vehicles to demonstrate this vision of gapless surfaces; however, they did not have the sensitivity, repeatability, or automotive reliability to move from concept to mass production. The vision existed, but the enabling technology did not.
Enter NextInput
NextInput, Inc., the leader in MEMS-based sensing, has quickly captured the attention of the automotive industry with its disruptive force sensing solutions. No longer are the industrial designers forced to make compromises or design trade-offs. NextInput’s patented sensing technologies are widely regarded as the only viable solution presently available that is both sensitive enough to detect intent though any material and robust enough to achieve AEC-Q100 automotive qualification. One innovative Electric Vehicle (“EV”) manufacturer has already gone to mass production replacing their steering wheel button controls with the NextInput solid state, ingress-free solution. In fact, several Automotive OEMs have already designed in NextInput solutions for a wide range of smart surface applications including steering wheels, 3D-Touch enabled displays, and door panel controls. NextInput is now working to bring contextual awareness to the automotive cabin.
Contextual Awareness
To truly replace the dashboard controls with a smart surface, contextual awareness is key in assisting the driver to get to the desired function quickly and intuitively. Like the smartphone, the modern automobile hosts a wide range of sensors. With inertial sensors, microphones, temperature sensors, GPS, infrared sensors, LiDAR systems, eye tracking, and now force sensing, information can be fused from the dozens and dozens of sensors available to detect, monitor, predict driver behavior, and control the outcome.
As a basic example, the car can detect when you are listening to music on your smartphone, transfer the music to the car stereo and bring up audio controls on the smart surface. Likewise, on a very cold morning, the car can automatically turn on the HVAC and display controls for the seat warmers. Now, imagine coming upon hazardous road conditions. The music is instantly muted and a map pops up to provide you with an alternative route. Through Artificial Intelligence and Machine Learning, the automobile will learn how to keep you safe and predict what you will want next.
Limitations of Voice
Some have predicted that smart surfaces will no longer be necessary with voice assistant being the replacement for traditional button controls. The flawed assumption is that the convenience of controlling the TV or the thermostat with one’s voice in the living room is directly applicable to the automobile cabin. However, while it is only a nuisance when the voice assistant does not understand your command in the home, this becomes a potential safety hazard while on the road. Linguistics have regional and educational differences that touch interfaces do not. Touch is the most intuitive of the human senses with children learning to interact with their world through touch long before they begin to speak. With voice, if the driver’s intent is not recognized the first time, or a child keeps changing the radio station from the backseat, the distraction inherently endangers both driver and those in close proximity. To ensure safety for critical functions, touch provides a more robust, intent-driven interface.
Semi-Autonomous and Fully-Autonomous Driving
An additional benefit of the sensor-driven contextual awareness in the car cabin becomes evident with the adoption of semi-autonomous driving technologies. The same sensors used to determine the driver’s intent can also be used to measure the health and wellness of the driver and judge incapacitation. For example, NextInput’s sensing solutions in the steering wheel and in the seat can be used to determine whether the driver is alert and actively engaged or slumped over and not in control. If an incapacitation is detected, the automobile can automatically take over, maneuver to safety, and call for help, instead of simply engaging the brakes because LiDAR detected an accident was imminent.
Even as fully-autonomous driving becomes more prevalent, the demand for smart surfaces will grow. As the cabin transforms into a productivity and entertainment space, consumers will demand an even higher degree of aesthetics as their interactions inside the cabin increase. The “touch and feel” experience of every interaction matters more now that their focus is no longer external. The car cabin of the future will have everything the passenger desires, and all within an arm’s reach.
Conclusion
With a tsunami of automobile cabins with smart surfaces currently in design, an entirely new Automotive HMI will come to market over the next several years, fundamentally changing the way consumers interact with their vehicles, from both luxury and economy. NextInput is on the forefront of enabling this paradigm shift to new industrial designs, having shipped over 20 million units to date of its production-ready force sensing solutions into Automotive, Mobile, and Consumer applications.
No Comments