Mercedes-Benz’s Virtual Assistant uses Google’s conversational AI agent
It's worth noting that Google Cloud's new " Automotive AI Agent " platform promises to "continue conversations and provide reference information" during users' journeys, and the first car announced to feature it is the new Mercedes CLA. That car has the next-generation MB.OS operating system with an enhanced MBUX virtual assistant.
It's worth noting that when Mercedes unveiled this at CES 2024, it didn't specify which of the company's LLMs it was running on. Meanwhile, the existing MBUX voice assistant system, which could handle around 20 commands triggered by "Hey Mercedes," now includes results provided by OpenAI's ChatGPT and Microsoft Bing, but it's not a conversational platform. Mercedes has indicated there are plans to implement this updated system in "more models" running the older voice assistant, but didn't specify which ones.
Everything about the Mercedes-Benz virtual assistant
It's crucial to note that the new MBUX virtual assistant will feature four "personality traits," including natural, predictive, personal, and empathetic. You can also ask it questions to gain more clarity and get what you need.
Google's new AI agent is custom-designed for automotive use and leverages Google Maps data to find points of interest, search for restaurant reviews, provide recommendations, answer follow-up questions, and much more. Google says that MBUX virtual assistant users will have access to near real-time Google Maps updates. It also states that the agent can handle complex, multi-turn dialogues.
The agent uses Gemini and runs on Google Cloud's Vertex AI development platform, designed to help businesses develop AI experiences.
January 13, 2025 – Mercedes-Benz and Google Cloud announced the expansion of their strategic partnership to introduce new conversational capabilities to the MBUX Virtual Assistant, powered by Google Cloud’s new Automotive AI Agent.
Built using Gemini on Vertex AI, Google Cloud’s Automotive AI Agent is specially tuned for the automotive industry and can reference information from Google Maps Platform to give users more detailed and personalized conversational responses about navigation, points of interest and more.
Google Maps Platform provides Mercedes-Benz owners with information about 250 million places around the world, and the map is updated nearly in real time, with over 100 million updates made to the map each day. With the enhanced search and navigation experience, users can converse naturally with the MBUX Virtual Assistant and get answers to questions like: “Could you guide me to the nearest fine-dining restaurant for a unique culinary experience?”. Users can also ask follow-up questions, like: “Does the restaurant have good reviews?” or “What is the chef’s signature dish?” and the MBUX Virtual Assistant can respond with accurate, up-to-date information and display navigation details through the vehicle’s native interface. The new experience will be available in the MBUX Virtual Assistant, with Mercedes-Benz’s new CLA series later this year.
Google Cloud’s Automotive AI Agent will also enable the MBUX Virtual Assistant to handle complex, multi-turn dialogue and can retain memory of conversations, which means users can continue conversations and reference information throughout their drives.
Mercedes-Benz’s virtual assistant, MBUX (Mercedes-Benz User Experience), has integrated Google's conversational AI technology to enhance its capabilities. This collaboration allows MBUX to provide more advanced natural language processing and understanding, making the in-car experience more intuitive for users.
With the integration of Google's AI, Mercedes-Benz aims to offer more natural and responsive voice commands, improving functions like navigation, media control, and personalized assistance. This enhancement enables the virtual assistant to better understand and predict user needs, creating a seamless and user-friendly experience.
Mercedes-Benz's latest MBUX Virtual Assistant, introduced in the new Mercedes CLA at CES 2024, incorporates Google Cloud’s Automotive AI Agent platform. This platform is designed to enhance the driving experience by supporting continuous, multi-turn conversations and referencing information throughout the journey.
Unlike the older version of MBUX, which could process around 20 voice commands (like “Hey Mercedes”) and relied on OpenAI’s ChatGPT and Microsoft Bing for search results, the new system is far more advanced. It’s built on Google Cloud's Vertex AI development platform and powered by Google's Gemini language model. The upgraded MBUX Virtual Assistant is capable of handling complex conversational queries, providing nearly real-time Google Maps updates, restaurant reviews, recommendations, and more. Its ability to process multi-turn dialogues means it can maintain context over multiple interactions, making it much more dynamic and intuitive.
The assistant's new design includes four distinct personality traits: natural, predictive, personal, and empathetic, enhancing its ability to offer more tailored, human-like responses. It also improves upon clarity by asking follow-up questions when needed to ensure accuracy in its responses.
Google CEO Sundar Pichai emphasized the transformational potential of these AI-driven "agentic" capabilities in the automotive industry, suggesting this is just the beginning of a more personalized, intelligent in-car experience. While the new system is being launched with the next-generation MB.OS operating system in the CLA, Mercedes plans to roll out this advanced assistant to additional models in the future. However, specific models haven't been named yet.
