TRAI

Advanced 3D perception with Time of Flight camera (TRAI)

Joint project with Analog Devices International.

The project is funded under the PTE technology transfer program of the Romanian Authority for Scientific Research, via UEFISCDI (project number PN-III-P2-2.1-PTE-2019-0367, contract number 23PTE from 03/06/2020). It takes place at the Automation Department of the Technical University of Cluj-Napoca, Romania, for a duration of 2 years and a total budget of 400000 EUR with 150000 EUR co-financing led by the project coordinator Analog Devices International (ADI) from the Romania.

Contact:

  • ADI (PI): Andrei.Cozma@analog.com, tel: 0751201039, Str. Teodor Mihali 64, Cluj-Napoca, Ro.
  • UTCN (P1): Levente.Tamas@aut.utcluj.ro, tel: 0264401586, Str. Memorandumului 28, Cluj-Napoca, Ro.

Abstract

Spatial perception and reasoning are becoming a must for an intelligent system, especially for autonomous systems. This can be achieved by means of special 3D cameras, which enable the shift from the 2D perception to the spatial one. As the 3D sensing technology is reaching maturity these cameras are becoming widespread and are currently being integrated into research platforms(e.g. mobile robots), industrial sector (e.g. automotive, AGV) or consumer level devices (GoogleTango, Kinect etc).

Analog Devices Romania (ADIRo) is a global leader in depth sensing and has one of the most advanced 3D camera technologies currently on the market. But, to enable customers to effectively integrate this technology into their products, ADIRo needs a re-deployable approach for developing depth sensing applications based on artificial intelligence. The custom-specific parts can be seen as transferring specific details to a generic AI system.

Driven by the current needs of ADIRo and relying on their expertise in spatial perception, researchers at the academic partner UTCluj will work on the following major objectives: (i)  performing keypoint-feature based 3D object recognition perception applications based on Artificial Intelligence (AI) and (ii) extending this approach towards transfer learning (iii) which is finally applied to customer-specific applications and finally. The involvement of B.Sc. and M.Sc. students is planned, together with the dissemination of the results in top conferences and journals. Besides this, at the economic impact, a multi-layered revenue system will ensure a market increase of the ADIRo in the European and global markets according to the business analysis model.  

The new AD-96TOF1-EBZ 3D camera platform available at ADIRo and the spatial perception and real-life application-driven skills at the UTCluj team are a perfect match for developing 3D perception applications thus raising the TRL of this product and broadening the market of the new ADI camera.

Objectives

O1. Extending the capabilities of the 3D camera from ADIRo with available open-source implementations of point cloud processing including deep learning-based variants as well;
O2. Integrating state-of-the 3D object recognition tools in a transfer learning framework based on a service approach for greater flexibility in low-volume client applications;
O3. Provide a set of customer-oriented use cases allowing the promotion of the innovative spatial perception services provided by the ADIRo.

Planned project activities

Phase 1: Development of interfaces and algorithms for the ToF camera

  • Hw&software interfaces, filtering
  • 3D object recognition PoC
  • Publications

Phase 2: Validation of the developed methods on the client side

  • Development of the transfer learning PoC
  • Know-how transfer and publication

Phase 3: Validation and dissemination of the methods

  • Validation of the PoC level products at the client side
  • Dissemination of the results

Outcomes of the planned activities

Phase 1: Development of interfaces and algorithms for the ToF camera – Report

Phase 2: Validation of the developed methods on the client side – Report

Phase 3: Validation and dissemination of the methods – Report

  • Developing client-oriented demos with the already added 3D perception modules
  • Transferring the accumulated know-how to ADIRo
  • Testing and validating together the results
  • Dissemination
  • Final workshop/future project tendering
  • AQTR2022, VISAPP, IFAC papers

Some relevant videos for the PoC demos:

List of relevant research papers within the project context (know-how) with pdf-s links:

Sumar în limba română:

Proiect finantat de catre UEFISCDI – Program PTE, proiect numarul PN-III-P2-2.1-PTE-2019-0367 cu finantare bugetara in valoare de 150000EUR si valoare totala de 400000EUR.

Utilizarea metodelor bazate pe inteligența artificială (IA) în sfera industrială devine în prezent o necesitate. Cu toate acestea, există doar un număr limitat de soluții profitabile și flexibile bazate pe AI pentru produsele cu un volum redus. Acest lucru se datorează costurilor relativ ridicate de integrare a soluțiilor avansate de IA în produsele personalizate ale clienților.

Impactul proiectului proiectului s-a realizat de-a lungul a trei axe majore: economică, științifică și educațională. Impactul economic s-a manifestat prin creșterea vânzărilor și profitului companiei. Cu privire la impactul științific, prin validarea algoritmilor dezvoltați, s-au publicat un număr semnificativ de articole la simpozioane internaționale. Aceasta au sporit atât vizibilitatea grupului de cercetare cât și nivelul colaborării cu industria în domenii emergente ale roboticii sau viziunii artificiale. Impactul educațional include beneficiile membrilor tineri ai echipei, în special masteranzi și doctoranzi, decurgând din participarea într-o echipă mixtă academico-industrială, aprofundând astfel atât baze teoretice, cât și o experiență practică.

Fig. Prototip cameră inteligentă dezvoltat de Analog Devices în cadrul proiectului PTE