Communication between machines and humans is hindered by the lack of shared awareness. To bridge this gap, Mitsubishi Electric Corporation, a leading global manufacturer of electronics and electric products, has reportedly developed a new technology called ‘Scene-Aware Interaction’, based on its proprietary Maisart®* AI technology.

Sources cite that by analyzing the scene based on multimodal sensing information, such as videos and images captured with cameras, localization information collected with LiDAR, audio information recorded with microphones, the system provides intuitive interaction using natural language.

This system identifies objects and understands events in the scene based on multimodal information. Then, the system creates natural sentences for guidance based on those events and objects.

To illustrate the technology, the company demonstrates a system that provides route guidance to a driver based on the awareness of the scene. A traditional navigation system would tell the driver to turn right in 100 meters. As this data is not so informative, drivers may not have an accurate sense of the distance.

Mitsubishi’s scene-aware interaction technology enables the navigation system to behave like a knowledgeable passenger in the car, describing the route intuitively using other cars and objects in the scene. It gives instructions like ‘follow the black car to turn right ‘, ‘turn left before the postbox’ to describe the route.

Moreover, the system also creates voice warnings, like ‘a grey car is coming from the front’ when objects in close proximity are expected to intersect with the way of the car.

Mitsubishi said in the statement that scene-aware interaction technology is anticipated to have a wide range of applications including machine-human interfaces for in-vehicle infotainment, interaction with robots in factory and building automation systems, monitor the health of the people, surveillance systems, support touch fewer operations in public, and many others.

Source Credit - https://www.mitsubishielectric.com/news/2020/0722.page