MimeTIC (team.inria.fr/mimetic/) and Morpheo (team.inria.fr/morpheo/) have a joined three-years PhD grant « Realistic Pose-Motion Transfer ». Please forward this information to any student who would be interested in such an experience.
PhD could take place either in Rennes (MimeTIC) or Grenoble (Morpheo) in France.
Please contact me for more information if needed.
Context: This PhD is part of the AVATAR INRIA project, a collaborative project between several INRIA teams with the aim to significantly advance the field of AVATAR modeling in particular by improving their realism. The PhD will be shared between the Mimetic team in Rennes, specialized in animation and the Morpheo team in Grenoble, specialized in moving shape capture.
Job Description: One of the objective of AVATAR is the ability to transfer the motion captured from a user to its avatar in a faithfull way. One of the main problems is that avatars do not perfectly look like the user (morphology variability) and may interact with a different environment (environment variability). Hence, a motion is in practice not limited to joint angles that model mainly the pose, and as provided by traditional mocap systems, but involves contextual information, such as relation in-between body surfaces and with the environment. This is especially true with contacts between body parts that cannot be captured with joint angles only. In order to better model human pose, a set of works consider the “interaction mesh” [Ho10, Bernardin17], a graph structure that connects joint centers and can be used to preserve distances between these centers when transfering body poses to an avatar. Interaction graphs aims at capturing the contextual information linked to the motion. However, while better preserving the interaction between body parts, the interaction mesh is still unable to accurately capture and transfer body surface information. The purpose of this PhD is to investigate innovative solutions that consider shape surface information instead of pose only information.
A first direction we want to explore is the extension of the interaction mesh to body surface information. This can be done through a graph connecting mesh vertices instead of joint centers, such as a tetrahedral graph. Several challenges must be faced for that. First, shape representations must be topologically consistent among subjects and poses in order to be able to implement an interaction graph structure that transfers information at the vertex level. Second, an approach must be designed to transfer body poses through such interaction graph structures. As for previous works with joint centers, this can be first experimented with an optimization framework that finds pose parameters subject to body constraints from the interaction graph.
A second direction in the longer run will be to investigate how learning could benefit to the process explain before, i.e. replacing the optimization with a data driven approach when transferring a captured pose to an avatar. Existing human datasets, such as the caesar dataset, enable relationships between body poses to be learned over different subjects. While this has been already largely studied in the literature, our objective here will be to study such relationships in the context of an interaction structure such as the interaction mesh or the interaction graph proposed before, to better capture and simulate the contextual meaning of the motion.
Expertise of the PhD candidate
The candidate should have a strong expertise at least in one of the following domains: computer geometry, computer animation, machine learning, optimization.
He or she will have to code demos and prototype and should consequently have skills in programming (Matlab, C++). A knowledge computer graphics tools (such as Maya, Unity, or 3DS Max would be useful).
[Bernardin2017] A Bernardin, L Hoyet, A Mucherino, D Gonçalves, F Multon (2017) Normalized Euclidean distance matrices for human motion retargeting. Proceedings of the Tenth International Conference on Motion in Games, 15
[Ho2010] ESL Ho, T Komura, CL Tai (2010) Spatial relationship preserving character motion adaptation. ACM Transactions on Graphics (TOG) 29 (4), 33