Incredible Breakthrough in 3D Face Scanning

By on September 20th, 2017 in research, Service

Tags:

 A terrific 3D model generated solely from a single 2D image
A terrific 3D model generated solely from a single 2D image

UK researchers have developed a very effective method of producing reasonably accurate 3D models from a single, 2D image. 

This is perhaps the holy grail of 3D face scanning: the ability to obtain a 3D model from only a single image. Traditional approaches to producing 3D facial models are far more complex. 

Two approaches are conventionally used: a 3D scanner, which projects some form of light on a subject. The light is shone from different directions, with the scanner software interpolating the individual detected point locations to gradually build an accurate 3D model. This approach is usable only by those who have access to a likely expensive 3D scanner and the training to do so. 

The other, more common approach, is to use a photogrammetric method in which a series of still images of a subject are analyzed. Images, typically numbering 40 or more, are taken from  different positions, up to 360 degrees around the subject. These images are then analyzed by software to see how the image structures shift position relative to the background as different images are examined. This enables the software to identify individual points in 3D space that are then converted into a usable 3D model. 

However, that approach requires very careful and comprehensive lighting and a suitable background containing contrasting portions that can be tracked by software. 

While possible, both of these methods are well beyond the access by non technical folks. 

Now thereā€™s a big change as the UK researchers have created a different approach. Hereā€™s how it works: a single image, usually a front-facing ā€œpassport styleā€ image is input. This image is run through a convolutional neural network, a software construct that is sometimes used in artificial intelligence and machine learning disciplines. The ā€œCNNā€ has been trained to match 2D images against a large library of existing 3D facial models. 

It seems they are leveraging the fact that faces donā€™t truly differ significantly in structure; only minor variations really exist. Therefore, this particular implementation would not work if 3D scanning other objects that are not human faces, for example.

However, that doesnā€™t stop anyone from using the same technique and training the system to produce 3D models of other types of subjects that donā€™t vary too much in geometry, such as dogs, automobiles, airplanes, etc. 

There is a website where you can actually try out this technology by uploading a single facial image. Iā€™ve tested a number of images and found the results to be surprisingly good. From this original image:

 Original 2D image of a happy fellow
Original 2D image of a happy fellow

This 3D model was generated within seconds:

 The corresponding 3D model is quite lifelike
The corresponding 3D model is quite lifelike

In fact, the resulting 3D models (which are provided in .OBJ format so that color texture information is included). This means you should be able to 3D print these models immediately. 

If you happen to have access to a full RGB color 3D printer, you can actually 3D print these models in full color. I think Iā€™ll be trying this soon. 

Of course, this is only an interesting research paper at the moment, but I suspect it could easily be turned into a commercial tool that can efficiently recognize a series of image types, beyond mere faces.

Also, the demonstration site may not persist forever and may disappear at any time. However, the researchers have also provided the code on GitHub so you can try this yourself. However, to make it work you will no doubt require a large database of 3D facial models to get it going. 

Via University of Nottingham, PDF and GitHub

By Kerry Stevenson

Kerry Stevenson, aka "General Fabb" has written over 8,000 stories on 3D printing at Fabbaloo since he launched the venture in 2007, with an intention to promote and grow the incredible technology of 3D printing across the world. So far, it seems to be working!