Non-rigid structure from motion

A realistic, stop-motion animatied dataset with dense ground truth.

Evaluating Non-rigid feature matching and Non-Rigid Structure From Motion (NRSfM) algorithms in a quantitative manner has proven to be problematic. The central problem of providing reference ground truth for comparison has been approached from many different angles. Synthetic, rendered imagery and MOCAP data both provide such reference, but fail to capture the true complexity and variety of realistic NRSFM deformations. Therefore, there is a lack and a need for a real-life NRFSM sequence with a dense, accurate ground truth. That need will be addressed in this project.

About the project

We wish to simulate motion using a stop-motion like animation technique. Here a non-rigid object is moved into a certain pose, an image is taken, the object's pose is altered slightly, another image is taken and so on. The result is a sequence that, when played at an interactive frame rate, provides the illusion of motion. We will apply the same principle here, in generating a benchmarking data set for NRSFM with a twist. That being in addition to acquiring an image, we also scan the object using a structured light scanner. This way, we acquire an image sequence which can be fed to a NRSFM algorithm and a 3D reference which can be used to benchmark the result. As an additional bonus, the stop-motion technique gives us great freedom to produce many different scenes that may provide varied challenges for NRSFM algorithms. Examples of these would be tearing, deformation, cracking etc.

Publications

Instrumentation for Estimating Surface Radiometry [2021]

M. E. B. Doest
Department of Applied Mathematics and Computer Science, Technical University of Denmark

A benchmark and evaluation of non-rigid structure from motion [2021]

S. H. N. Jensen, M. E. B. Doest, H. Aanæs, A. Del_Bue
International Journal of Computer Vision , 129(4), 882-899

A benchmark and evaluation of non-rigid structure from motion [2018]

S. H. N. Jensen, A. Del_Bue, M. E. B. Doest, H. Aanæs
arXiv:1801.08388v2

Our 3D vision data-sets in the making [2015]

H. Aanæs, K. Conradsen, A. Dal_Corso, A. B. Dahl, A. D. Bue, M. E. B. Doest, J. R. Frisvad, S. H. N. Jensen, J. B. Nielsen, J. D. Stets, G. Vogiatzis
Conference on Computer Vision and Pattern Recognition 2015 , Institute of Electrical and Electronics Engineers

Accuracy in robot generated image data sets [2015]

H. Aanæs, A. B. Dahl
Image Analysis (Proceedings of SCIA 2015) , Springer, Lecture Notes in Computer Science, vol. 9127, 472-479