Alexandre Donciu-Julin | Technical Animator | Mocap Director

Howdy

As a mo-cap animator, I was invited this week to an Intel® RealSense™ Hands-on Lab in Berlin. I had the opportunity to discover this new camera and the astonishing features it's offering. Most of all, I was interested in its face-tracking capabilities: eyes, cheeks, shape of lips, expressions, emotions, and so on. Indeed, paired with a motion-capture software like Faceshift, you can get full facial animation and lipsync without any markers needed, as opposed to an optical system.


Faceshift is a very powerful software, but it rely entirely on the camera you are using. I tested it some time ago with the first version of the Kinect sensor, and the results were pretty good. I couldn't wait to see what this one has to offer! We were given a few hours to develop an prototype application using the RealSense camera features.


No app for me, since I'm not from the dark side of programming ;) But I used the time to install a demo of Faceshift and set-up the camera for it. A quick scan of my head transformed a generic head into a 3D model looking disturbingly like me! I refined it by scanning my face, while performing various expressions (BlendShapes): mouth open, frowning, puffed cheeks... I let you imagine the startled programmers looking at me all around! Once finished, I retargeted the face tracking on a 3D head and voilà! Full face tracking in real-time, impressive! I played along with the camera parameters to get the best result possible. And what a result! The high-quality tracking delivered by the camera, paired with Faceshift algorithms met all my expectations. It took me less than an hour to get some results matching industrial standards!


I did not win the top price this day, which was a laptop with an integrated RealSense camera. But still, I was glad to impress everyone with my results and even got a consolation prize, by posting this devilish selfie on Twitter! ;)


A very productive day for sure, I'm really glad I was invited. And I kept the best for last: I was allowed to keep the RealSense camera, so get ready for more exciting projects!