AI and the personalized car

If CES 2017 was dominated by vehicle technology announcements, then the biggest story was the rise of AI and intelligent personal assistants (IPAs) in vehicles. Announcements included the use of Amazon’s Alexa, Google’s Assistant, Microsoft’s Cortana and other, in-house projects. In the last few weeks Audi, Ford, Hyundai, Nissan, Toyota and Volkswagen all announced some form of new social robotic assistance, while BMW had previously announced it was working with IBM to test the Watson AI in four BMW i8 vehicles.

IPAs are voice-assisted software agents, while the term AI is mostly used as a marketing term for IPAs that make great use of deep learning. IPA technology has improved fast and they are already present in many consumer technology products, and people are already familiar with using them in smartphones. The advantage for automotive OEMs of using other companies’ IPAs and AI features is obvious. They do not have to spend the time and effort developing their own, and, given how competitive this market is, it can offer users an additional feature. The gap between what AI and IPAs can achieve today and the vision for them as articulated by automotive OEMs is large but will narrow over time.

Alexa steals the show

Amazon has had remarkable success with Alexa in recent months. First launched in November 2014 with Amazon’s Echo smart speaker, it now has more than 7,000 “skills” (built-in app capabilities) and is either integrated or directly built into dozens of consumer electronic products. Indeed, Alexa appeared in around 30 product announcements at CES 2017 as a built-in or integrated feature. Ford announced at CES 2016 that it was exploring ways to integrate Alexa into its vehicles. At CES 2017, it announced integration into its SYNC 3 AppLink software. SYNC 3 is the infotainment system used across Ford’s 2017 line, which also includes Android Auto and Apple CarPlay. In November 2016, Hyundai announced it had integrated Alexa into its Blue Link connected car app. And at CES 2017, Volkswagen announced Alexa would be integrated into its Car-Net system.

Much of Alexa’s success is down to Amazon’s strategy of giving the IPA away for free, as Google did with its Android operating system. Amazon is hoping to gain ground quickly, by tying users to its ecosystem and enabling them to use the entertainment services, apps and shop in a new space – the vehicle – with Alexa as the UI.

Announced at CES 2017 was the SmartDeviceLink (SDL) Consortium, an open-source project begun by Ford and designed to enable other OEMs to use the software to link smartphone apps with their own connectivity systems. Amazon has joined the consortium alongside several automotive OEMs: Mazda, PSA, Subaru, Suzuki and Toyota, and several key suppliers, and aims to expand the opportunity of Alexa in the car. Doing so may bring even more partnerships down the line with these automotive OEMs.

BMW and Renault-Nissan elect for Microsoft’s Cortana

Microsoft has had automotive infotainment and connectivity solutions for many years, with mixed success. But now it is embarking on a new strategy with the Microsoft Connected Vehicle platform, a set of services built on the Microsoft Azure Cloud, available to automotive OEMs to create connected car solutions. The system is designed to collect sensor and usage data, providing automotive OEMs with ways to build services alongside that data. One of those services is the Cortana IPA. At CES 2017, BMW and Renault-Nissan announced they will make use of a number of the services. Volvo will add Microsoft’s Skype for Business app to its new 90 series vehicles and is exploring implementing Cortana.

Microsoft’s main benefit is its vast range of cloud services, storage and the experience it can offer. The company has long-term plans to integrate Cortana into a wide range of consumer products, which should help expand the Cortana/Microsoft ecosystem.

Google’s rebranded Assistant is slow to get going

Google has rebranded its IPA from Google Now to the Google Assistant, which is now in millions of Android smartphones and heading to Android TV and Android Wear devices.

While there have been some Google Assistant announcements, the partnerships are not as expansive as with other IPAs. Daimler has announced that it will integrate the Google Assistant on Google Home with Mercedes-Benz vehicles in early 2017, allowing for minor communication between the vehicle and the IPA. But it is not directly integrated and as such offers little value to the customer. At CES 2017, Hyundai announced it will integrate the Google Assistant with its Blue Link app, as it had done with Alexa. With Waymo seeking to provide its platform to automotive OEMs, Google services must be tightly integrated, so it is important Google does not lose momentum with its IPA in vehicles.

Toyota’s vision of AI as a handover solution

Toyota’s Concept-i vehicle showcased at CES 2017 featured “Yui”, an AI copilot. Created by Toyota’s Calty Design Research center in California, Yui is not only designed to offer personalization benefits, but also to keep drivers safely engaged during autonomous driving, ready to take control of the vehicle when required. It is even designed to use biometric sensors throughout the car, to detect and analyze the driver’s emotional state. Having an AI in the vehicle can help keep the driver attentive through verbal cues and ensure the handover process is as seamless as possible.

Nvidia is another company involved with AI in vehicles. It aims to work with Audi to create the “World’s most advanced autonomous vehicles”, due to hit the roads in 2020. Using the term “AI car”, CEO Jen-Hsun Huang said on stage at the CES 2017 keynote that the vehicles were demonstrating AI skills “not end of this year, not end of next year… right now.” Nvidia also announced a partnership with Mercedes-Benz to bring an AI car to market in 2018, using the same platform. But the end products will certainly be very different, with Mercedes likely to use the technology to personalize the driving experience. The company has already developed this area with the Drive PX2 AI computing platform and Xavier AI supercomputer, but very little information on the nature of the AI itself has been announced. A similar AI solution to Toyota’s was revealed, the Nvidia AI copilot, which will monitor driver concentration, though it will probably provide simple warnings and is still in development.

Apple bides its time with Siri

Apple’s Siri assistant is already available in a range of cars, including models from BMW, Nissan, Hyundai and Ford. These use Apple CarPlay software, though require an iPhone connection. CarPlay is supported in 200 announced car models. Given rumors that Apple’s Project Titan has moved to focus more on software, it may be that Apple begins to position a more comprehensive Siri-based assistant as an offering to automotive OEMs.

AI needs to offer value beyond additional services

For AI and IPAs to provide real, intelligent assistance to passengers and drivers, they must be able to provide information and advice beyond updates on weather or restaurant locations. Toyota’s vision comes closest to what IPAs/AIs will look and sound like in the future. It is likely that they will become far more integrated into vehicles and assist with autonomy. Voice recognition still struggles to be as useful as it could be. Though speaking tends to be faster than typing, ensuring grammar and syntax are all correct afterwards may reduce this time benefit. Additionally, there are many occasions where using voice commands is unhelpful, such as when around other people.

AIs must also be able to give answers to complex contextual questions, such as, “How long does it take to drive to Houston?”, followed by, “What’s the weather like there?” to give the weather for Houston, rather than the driver’s current location. AIs must also be able to answer questions while still responding to previous ones, without having to pause to give the user a prompt. Nuance’s Dragon Drive multi-passenger interaction solution helps identify and isolate speech from individuals in the vehicle and its “barge-in” technology allows for corrections and interruptions. Solutions like these are vital for AI to have a meaningful impact on future autonomous travel. Finally, given the range of different IPAs available on a wide variety of consumer devices, automotive OEMs must ensure that a range of alternative AIs can be offered to avoid frustrating customers. The benefit of an IPA is lost to the consumer if the valuable user information is not available for sharing across multiple devices and vehicles.