Augmented reality

Our goal is to develop vision-based methods which allow significant progress of AR technologies in terms of ease of implementation, usability, reliability and robustness. Matchin interest points, pose computation and visual reconstruction are our two main research topics. Our originality lies in our in-large approach of the AR applications which leads us to consider all the components of an AR system and various means, ranging from autonomy to interactivity, to reach the robustness to space and time scaling required for the applications to be effective.

Urban Augmented reality

  • Facade proposal for urbain augmented reality. ISMAR 2017. pdf

    We introduce in this paper a novel object proposals method specific to building facades. We define new image cues that measure typical facade characteristics such as semantic, symmetry and repetitions. They are combined to generate a few facade candidates in urban environments fast.

A contrario methods for robust point image matching

  • Robust point image matching.

    Our aim is to design new algorithms for feature matching with structure and motion applications in view. We have proposed matching algorithms which are robust to wide viewpoint changes and to repeated patterns. We have investigated statistical models (namely a contrario models) to this aim. The advantage of such models is that detection thresholds are automatically derived from images statistics. We have especially proposed a new a contrario matching algorithm incorporating both geometric and photometric constraints.

  • Robust image matching based on viewpoint simulation techniques.

    Matching techniques based on SIFT features suffer from a lack of robustness with respect to strong viewpoint changes. Viewpoint simulation technniques as Affine SIFT ASIFT are a powerful way to improve viewpoint invariance. We have recently improved ASIFT by incorporating the aforementioned a contrario model which aggregates set of correspondences consistent with a local homography. Our algorithm performs better than any existing generic point matching algorithm. How to overcome perceptual aliasing in ASIFT, ISVC 2010


Enhancing pose estimation with view synthesis

Given a model acquired with Structure from Motion, with each 3D point represented by a cluster of 2D descriptors extracted from images, our goal is to enrich the cluster with descriptors computed from virtual viewpoints poorly represented in the original sequence. Amethod for optimal positionning of the virtuals views has been proposes at BMVC 2016. This work was the object of P. Rolin's PhD thesis.

  • Simulation de point de vue pour la localisation à partir d'un modèle non structuré, RFIA 2014 pdf
  • Viewpoint simulation for camera pose estimation from an unstructured scene model, ICRA 2015 pdf
  • Simulation de point de vue pour la mise en correspondance et la localisation Traitement du signal pdf
  • Enhancing pose estimation through efficient patch synthesis, BMVC 2016 pdf

Red: sequence used by SFM, green: virtual viewpoints

test image pose without synthesis (100 trials) pose with view synthesis (100 trials)

Augmented reality for multi-planar environments

  • An AR system for multi-planar environments: This work was done in the context of the European project ARIS. See methods and results as well as a video demonstrating the final system.
  • Use of model selection to improve the statility of the pose, Vision Video and Graphics 2003 pdf
  • Use of Inertial Sensors to Support Video Tracking, published in Computer animation and virtual worlds pdf
  • Tracking-by-Synthesis Using Point Features and Pyramidal Blurring, ISMAR 2011 pdf

Interactivity in AR systems

For some applications, human can be relied upon to acquire structured scene models. Our motivation is to enable fast capture of textured surface models suited to AR. We are investigating various methods for in-situ sketching using a video camera as an interaction device.

  • Reconstruction et augmentation simultanées de scènes planes par morceaux : papier RFIA 2008 pdf and video.
  • Detection of the Intersection Lines in Multiplanar Environments: Application to Real-Time Estimation of the Camera-Scene Geometry, ICPR 2008 pdf . An extended version of this work has been published in 2011 in The Visual Computer pdf
  • Immersive image-based modeling of polyhedral scenes, ISMAR 2009 pdf See also the associated video.
  • In-Situ 3D Sketching Using a Video Camera as an Interaction and Tracking Device, EUROGRAPHICS 2009 pdf
  • Interactive building and augmentation of poecewise planar environments using the intersection lines, The visual Computer, 2011, 27(9) pdf
  • In-Situ Interactive Modeling Using a Single-Point Laser Rangefinder Coupled with a New Hybrid Orientation Tracker, ISMAR 2013 pdf
CNRS INRIA Université de Lorraine LORIA

copyright INRIA / Photos C. Lebedinsky