ROS2AR

The main focus of the ROS2AR FTP is the integration of AR facilities into ROS-I by means of object recognition techniques in a production specific environment using smart glasses. The user is assisted in the object detection using landmark and 2D object detection in the HRI context. The detected object related information (e.g. digital twin) is gathered and shared to the user using a ROS bridge and is visualized in an augmented manner by means of notifications using hugely popular Android ecosystem with direct application for head mounted devices (such as smart glasses).

This FTP is addressing the seamless integration of ROS-I with the wide variety of Android based ARemerging devices, with a special focus on head mounted devices.

Through this integration, the user will be able to receive real-time notifications about the environment, such as the state of the equipment nearby or other object specific data, like the digital twin of the object or equipment proprioceptive state in the field of view of the AR device. This enables the user to operate hands free, while at the same time receiving context relevant information on demand.

This FTP opens the ROS-I community to interface with the most popular mobile operating system globally (Android) which covers more than 75% of the market. This is the technical need the FTP is addressing.

We are addressing the Digital Twin use case in particular, where the user obtains live context-relevant technical data about an equipment as an augmented representation of its state with examples such as information about the specific operation the equipment is doing as part of its execution routine, scheduling information about the work queue of the equipment or monitoring info about abnormal or exceptional events.

The solution, with all work being public, can serve as a baseline for other use cases from the above list, enabling the community to re-use the work this FTP will deliver.