Your smartphone as a 3D scanner

Scientists from the Computer Vision and Geometry Lab of ETH Zurich developed an app that turns an ordinary smartphone into a mobile 3D scanner. Instead of taking a normal photograph, a user simply moves the phone around the object of interest and after a few motions, a 3D model appears on the screen.

Smartphone with 3D scanner  
The new software works with existing smartphone technology that allows the user to scan a 3D model almost as easy as taking a photograph (Photo: Institute for Visual Computing / ETH Zurich).

3D scanning aims to capture the geometry of the 3D world. However, most existing solutions require a complicated setup, are often hard to use and might not always work outdoors. Marc Pollefeys, professor at the Institute for Visual Computing  and his group found a way to develop a software that works with existing smartphone technology that allows the user to scan a 3D model almost as easy as taking a photograph. The technology was demonstrated publicly for the first time today at the International Conference on Computer Vision in Sydney, Australia .

Instead of taking a normal photograph, a user simply moves the phone around the object of interest and after a few motions, a 3D model appears on the screen. As the user keeps moving, additional images are recorded automatically and used to extend the 3D model. As all calculations are performed directly on the phone, the user gets immediate feedback and can select additional viewpoints to cover missing parts of the 3D model. “This is an important advantage compared to solutions that batch process all the images in the cloud at a later time”, explains Marc Pollefeys.

Approach works in a wide variety of settings

The app also makes it possible to visualise the state of the 3D scan from different viewing angles, allowing the user to cover all the areas of the object he is interested in. Having a solution on a mobile phone enables the acquisition of 3D scans on-the-fly anywhere. The approach works in a wide variety of settings, including low-light conditions such as inside a museum. A user can capture a 3D model of a museum piece and interactively study it at home later. After reviewing the model, a user might decide to upload his 3D data to a cloud service to further refine the results.

By using the inertial sensors of the phone, the scanning process can be made simple, intuitive and robust. After the 3D capture is started, the system automatically determines the correct moments to extract camera images based on the user motion. “Only two years ago, such a software only run on massive computers. We were able to shrink processes down on smartphone level and make them highly efficient”, says Marc Pollefeys.

Increased interactivity for the user

Contrary to image-only 3D capture solutions the app is able to determine the absolute size of the scanned 3D object, as well as the vertical direction. Because of the complexity of the calculations needed to reconstruct hundreds of thousands of points, the graphics co-processor (GPU) of the phone is used to enable a faster reconstruction and increase interactivity for the user.

The technology also allows the 3D capture of faces, giving a third dimension to portraits, profile pictures or images of loved ones. Having a convenient way of getting 3D models of everyday objects, users will now be able to copy real-world objects by scanning a full 360 degree model of an object. The resulting 3D model can be used for visualisation or augmented reality applications, or even be used for 3D printing, potentially at a remote location, effectively enabling the user to replicate an object.

The patent pending technology was developed exclusively by ETH Zurich and can run on a wide range of current smartphones.

Smartphone as a 3D scanner
  • Smartphone as 3D scanner
    The approach works in a wide variety of settings, including low-light conditions (Photo: Institute for Visual Computing / ETH Zurich).
  • Scanned Statue
    A user can capture a 3D model of a museum piece and interactively study it at home later (Photo: Institute for Visual Computing / ETH Zurich).
  • 3D capture of faces
    The technology also allows the 3D capture of faces (Photo: Institute for Visual Computing / ETH Zurich).
  • Original and 3D print
    The resulting 3D model can be used for 3D printing (Photo: Institute for Visual Computing / ETH Zurich).

Leave a comment

We are interested in your comments. Please use our form to send your comment to us:

14 comments

  • Peter Borrfoirs December 5, 2013 17:13

    Very interesting product. When will it be available for download or purchase? And what will the price be?

     
     
  • Stéphane Mor December 6, 2013 03:25

    This looks great! Is it possible to give it a try? Do you have a newsletter to keep updated of the progress of the app? Thank you very much!

     
     
  • Paresh Panchal December 6, 2013 08:44

    Wow, great! I was thinking one day it would be reality. Best of luck. From where can I download. I am excited to use it.

     
     
  • Marc Wakefield December 6, 2013 11:41

    WOW! I need this app in my life. Very happy you seem to have created an Android version. I can not wait to try this on my Galaxy Note 3. Please get in contact if you need an additional beta tester.

     
    • Peter Rueegg, editor ETH-News December 6, 2013 14:12

      Notice from the editors: This app is not available publicly yet. Scientific work is ongoing. There's no release date known. We keep our readers posted.

       
       
     
  • Alfredo Faubel December 8, 2013 01:01

    Excellent, congratulations ! Can't wait for the app to be available.

     
     
  • Stewart Bowland December 13, 2013 01:56

    I'm a mechanical designer and I find I am constantly having to work with models of components in our designs that don't exist as a3d model that I can use for designing with. This application looks awesome and will definitely improve my ability to work with geometry that I otherwise would have to manually draw and likely make a poor representation of. Please let me know when the sp is available.

     
     
  • Uma Sankar December 13, 2013 16:45

    Excellent app. It will definitely change so many fields (especially Design and Arts) in the world.

     
     
  • Brett Barkman December 30, 2013 15:14

    I would also like to know when this app becomes available.

     
     
  • Mike January 20, 2014 22:39

    IS there anyway we can collaborate with your team?
    We are in the biotech industry focused on brain and cell imaging and reconstruction. Please refer me to the right party to contact with, thanks!

     
     
  • Lasse March 5, 2014 09:56

    Hi. I understand your app is not yet finished, but is it possible to launch a beta version? I have been looking for this kind of app for a long time, this could really change my workflow!

     
     
  • Robert Gorman March 31, 2014 06:52

    Definitely willing to try a Beta version!

     
     
  • Bob Delisle April 1, 2014 20:41

    Very exciting project! Where can I purchase this app and what specific phone(s) does it work on?

     
     
  • Nigel Alexander April 3, 2014 10:44

    We would welcome an opportunity to collaborate with you as we are implementing a number of advanced Image recognition systems using Smartphones. Please keep us appraised of Developments. If you are seeking further Beta testing our organisation would be very willing to co-operate.

     
     
 
 
Site URL: https://www.ethz.ch:80/content/main/en/news-and-events/eth-news/news/2013/12/your-smartphone-as-a-3d-scanner.html
24.04.2014
© 2014 Eidgenössische Technische Hochschule Zürich