PhD CANDIDATE – GEOMETRIC CHANGE DETECTION in 3D MODELS, MODEL UPDATE and APPLICATION to MIXED REALITY EXPERIENCES

InterDigital, Rennes, France

Contacts: , ,

.

Summary

This PhD will be carried out over three years in the context of a collaboration (CIFRE grant) between InterDigital Research & Innovation, located in Rennes, France, and the academic research lab Ambiances Architectures Urbanités (AAU), located in Nantes, France, and under the supervision of Prof. Guillaume Moreau (Research Director, Centrale Nantes, https://www.ec-nantes.fr/m-guillaume-moreau–57175.kjsp). Both labs have recognized track records on augmented reality.

The PhD aims at designing novel methods leveraging computer vision and deep learning techniques for detecting geometric changes in 3D models, and more precisely to identify and characterize the insertion, the removal or the displacement of a real object in the reference 3D scene.

With the increasing interest from professionals and consumers for Mixed Reality (MR) applications, the capture and 3D reconstruction of both indoor and outdoor scenes have regained a lot of attention, as it is a key component to realistic real-virtual interactions (occlusions, collisions, shadows, …). For example, interior design and furnishing is a typical example of Mixed Reality consumer indoor applications requiring an accurate model of the room; similarly, town planning and touristic site visits may benefit from up-to-date models of the outdoor environment. Best MR experiences require 3D models that perfectly match the scene at the moment of execution.

Although the precise reconstruction of a 3D model of a real scene may require preparation and be highly time-consuming, it is likely to be accepted if it can be done within a pre-processing step, before the actual execution of the MR application by the end user. However, a real scene, even said to be static, only stays fully static over a very short period of time. For indoor cases, a piece of furniture is moved, while for outdoor cases, a house is destroyed, a building is constructed… So, a 3D model created from observations at time t₁ will only match a portion of the same scene observed at time t₂. Hence, we first need quick and efficient methods for detecting geometric changes, then related methods for updating the 3D model.

The goal of the PhD will be to investigate new computer vision approaches, as well as relevant neural network architectures, for geometric change detection given a reference 3D model and RGB images depicting the current state of the considered scene. Recent advances in semantic segmentation should be leveraged to improve accuracy and robustness. Importantly, the solution must be close to real-time and run on a mobile device. Appropriate techniques to update the 3D model will also be proposed. The resulting algorithmic modules will be integrated into MR demonstrators showing the added value of change detection and model update for new MR experiences.

Responsabilities:

• Invent, implement and validate new technical concepts according to the team strategy and with the academic partner lab.

• Participate to the research valorization effort through invention disclosures, technology transfers, and publications in peered-review journals and prestigious conferences.

Qualifications:

• Master in computer vision or related field.

• Strong experience in SW development (python/c++).

• Background in Machine Learning / Deep Learning.

• Experience with 3D reconstruction and analysis, real-time processing, GPU programming and/or graphics rendering would be a plus.

• Team spirit, enthusiastic, motivated, and creative attitude.

• Good communication skills to promote activities internally and externally. Fluent English mandatory, french appreciated.

Location:

Rennes, France, with regular stays in Nantes, France.

About InterDigital

InterDigital develops mobile and video technologies that are at the core of devices, networks, and services worldwide. We solve many of the industry’s most critical and complex technical challenges, inventing solutions for more efficient broadband networks, better video delivery, and richer multimedia experiences years ahead of market deployment. InterDigital has licenses and strategic relationships with many of the world’s leading technology companies. Founded in 1972, InterDigital is listed on NASDAQ and is included in the S&P MidCap 400® index.

Within the InterDigital Immersive lab. in Rennes, France (https://www.interdigital.com/video-resources/), the PhD candidate will be part of a team developing techniques and solutions addressing the large domain of Mixed Reality and that already published its work and developed and demonstrated prototypes in top conferences and major trade shows.

About AAU Laboratory and CRENAU Research team

The Ambiances, Architectures, Urbanités (AAU) research unit brings together two teams on two different sites: the Urban Architecture Nantes Research Centre (CRENAU), located both at ENSA Nantes and Centrale Nantes, and the Research Centre on Acoustic Space and Urban Environment (CRESSON) at ENSA Grenoble. The Laboratory consists of about 125 members over the two sites (cf. organizational chart), among which about 1/3 are PhD students. The researchers and professors-researchers in the teams belong to different schools of architecture, to CNRS, to Centrale Nantes, and to other institutions via agreements.

Created in 2002, the Virtual and Reality Team at AAU is composed of a score of persons, mainly employed by Centrale Nantes. It focuses on Virtual and Augmented Reality. The main research themes in AR/MR are pose computation from local to large scale, 3D urban environment reconstruction and human factors. The team is associated to INRIA Hybrid team. The team publishes its work in the best scholar journals and conferences and has led several successful projects with companies.