copyright INRIA / Photos C. Lebedinsky

Augmented reality

Our goal is to develop vision-based methods which allow significant progress of AR technologies in terms of ease of implementation, usability, reliability and robustness. Matchin interest points, pose computation and visual reconstruction are our two main research topics. Our originality lies in our in-large approach of the AR applications which leads us to consider all the components of an AR system and various means, ranging from autonomy to interactivity, to reach the robustness to space and time scaling required for the applications to be effective.

A statistical framework for robust point image matching

  • Robust point image matching

    Our aim is to design new algorithms for feature matching with structure and motion applications in view. We have proposed matching algorithms which are robust to wide viewpoint changes and to repeated patterns. We have investigated statistical models (namely a contrario models) to this aim. The advantage of such models is that detection thresholds are automatically derived from images statistics. We have especially proposed a new a contrario matching algorithm incorporating both geometric and photometric constraints.

    • Fundamental matrix estimation without prior match, ICIP 07. Pdf file ,
    • Computing the uncertainty of the 8 point algorithm for fundamental matrix estimation, BMVC 2008. Pdf file
    • Determining point correspondences between two views under geometric constraint and photometric consistency, INRIA report et papier RFIA 2010
    • An a contrario model for matching interest points under geometric and photometric constraints. SIAM Journal on Imaging Science 2013. Pdf file

Red: sequence used by SFM, green: virtual viewpoints

  • Robust image matching based on viewpoint simulation techniques.

    Matching techniques based on SIFT features suffer from a lack of robustness with respect to strong viewpoint changes. Viewpoint simulation technniques as Affine SIFT ASIFT are a powerful way to improve viewpoint invariance. We have recently improved ASIFT by incorporating the aforementioned a contrario model which aggregates set of correspondences consistent with a local homography. Our algorithm performs better than any existing generic point matching algorithm.

    • How to overcome perceptual aliasing in ASIFT, ISVC 2010. Pdf file
  • Improving the initialization stage with simulation:

    Given a model acquired with Structure from Motion, with each 3D point represented by a cluster of 2D descriptors extracted from images, our goal is to enrich the cluster with descriptors computed from virtual viewpoints poorly represented in the original sequence. This method dramatically improves the reliability and the accuracy of pose estimation for distant views.

    • Simulation de point de vue pour la localisation à partir d'un modèle non structuré , RFIA 2014 Pdf file
    • Viewpoint simulation for camera pose estimation from an unstructured scene model, ICRA 2015 pdf
    • Simulation de point de vue pour la mise en correspondance et la localisation pdf
    • Enhancing pose estimation through efficient patch synthesis, BMVC 2016, pdf

Augmented reality for multi-planar environments

  • An AR system for multi-planar environments: This work was done in the context of the European project ARIS. See methods and results as well as a video demonstrating the final system.
  • Use of model selection to improve the statility of the pose, Vision Video and Graphics 2003, Pdf file
  • Use of Inertial Sensors to Support Video Tracking, published in Computer animation and virtual worlds Pdf file ,
  • Tracking-by-Synthesis Using Point Features and Pyramidal Blurring, ISMAR 2011 Pdf file

Interactivity in AR systems

For some applications, human can be relied upon to acquire structured scene models. Our motivation is to enable fast capture of textured surface models suited to AR. We are investigating various methods for in-situ sketching using a video camera as an interaction device.

  • Reconstruction et augmentation simultanées de scènes planes par morceaux : papier RFIA 2008 Pdf file and video .
  • Detection of the Intersection Lines in Multiplanar Environments: Application to Real-Time Estimation of the Camera-Scene Geometry, ICPR 2008 Pdf file . An extended version of this work has been published in 2011 in The Visual Computer Pdf file .
  • Immersive image-based modeling of polyhedral scenes, ISMAR 2009, Pdf file . See also the associated video .
  • In-Situ 3D Sketching Using a Video Camera as an Interaction and Tracking Device, EUROGRAPHICS 2009 Pdf file
  • Interactive building and augmentation of poecewise planar environments using the intersection lines, The visual Computer, 2011, 27(9) Pdf file
  • In-Situ Interactive Modeling Using a Single-Point Laser Rangefinder Coupled with a New Hybrid Orientation Tracker, ISMAR 2013 Pdf file