Face2Face: Real-time Face Capture and Reenactment of RGB Videos (CVPR 2016 Oral)

Founder here Maybe – one of my co-founders is Prof Matthias Niessner who's been behind a large chunk of the seminal and widespread research in this space. [1] ▶ this video [2] link

Founder here. AMA :) To answer a few recurring questions in the thread ---> Use case. Video is a way more effective way to communicate than text. Not for the HN crowd, but if you're a blue collar worker a 2 minute video in your native language is much preferred to a 5 page pdf for training. Anyone who has tried to record a simple corporate video know the pain of cameras, film crews, 25 takes to get one that works and post production. Cumbersome, slow and multidisciplinary. By the time the video is done the content is out of date. Synthetic video is not yet at the quality of real video. Eventually it will be. But the mistake many are making here is comparing it to real video; it should be compared with text. In X years we'll be able to make Hollywood films on a laptop without needing anything but time and imagination. Just like we can digitally compose music in Ableton, create images in Photoshop and type novels on keyboards rather than with pen and paper. My (obviously biased;)) belief is that synthetic media will eventually become foundational technology that will move media production from cameras/microphones to API's. We'll be able to do all kind of things we couldn't do before. Eg. personalized and interactive rich media, video-driven chatbots and eventually Hollywood blockbusters made by your favourite YouTuber from his or her bedroom. ---> Uncanny valley Simulating real video is incredibly hard. We're constantly improving and launching more expressive synthesis soon. Fr

▶ this video Face2Face would defeat this: anyone with enough images of the victim (and copy of ID) could impersonate them in this scheme.

Photo, audio, and video evidence should already be dismissed until one is able to verify the integrity and source. All of these can already be believably faked - it's just a matter of educating people that a layperson can easily create fake things by using tools developed by research teams. Fake text is the easiest to fake if you can identify the font used - any image editor will work. HN uses 9pt Verdana, even without using dev tools I could fake your post to say anything I wanted it to say since it would just be 9pt Verdana on a solid background set to text wrap every 1050px. See: ▶ this video & link

Here's another scary GAN proof of concept [0]. In this case, researchers transferred someone's face in real time to facial expression and mouth movements of public figures. Combined with DeepMind's new tech that seems to be able to produce human voice with believable candor and inflection [1], you could make some very convincing fake footage. [0] ▶ this video [1] link

We're already at the stage where one can't trust the video evidence, unless it's backed by the camera's signature/encryption. Creating a very realistic fake is now trivial: ▶ this video link Google is very close to synthesizing realistic voice. It's game over, as far as I can see. It's a matter of time someone creates a fake video of someone famous saying something very outrageous, like nazi propaganda, and it will result in the destruction of that person's career and life. We really need something like Secure Enclave in every camera. EDIT: Another related video: link

> Seems from right out of the gate, they are breaking their own ethical guidelines as a cheap promotional tactic. If they care that little about themselves and a former president of the United States, what do they care about your likeness. We state in our blogpost that we make an exception for Obama/Trump in order to raise public awareness. Both of them are regularly used in Machine Learning benchmarks (for example [0] [1]). Note that we don't allow users to generate from Trump/Obama's voice. Once again, we care a lot about these issues and that's why we only allow users to copy their own voice. [0] link [1] ▶ this video These issues are challenging and suggestions about how you think the technology should be introduced/regulated are very welcome.

This article title is a bit off. The audio is not generated, the video is not convincing, and the original conversation is not fake. This is an art project which highlights the implications of research like Face2Face: ▶ this video

Face2Face: Real-time Face Capture and Reenactment of RGB Videos (CVPR 2016 Oral) ▶ this video And I agree, I find this much more verosimil.

Combined with Face2Face[1] live video impersonation, it is truly time to be very careful verifying videos or even live streams. ▶ this video

> "Fake News" is about to get a lot more compelling hen you can make anyone say anything as long as you have some previous recordings of their voice. Adobe has already developed that technology: link Now imagine combining it with this: Face2Face: Real-time Face Capture and Reenactment of RGB Videos ▶ this video Perhaps using the intonation from the face-actor's voice to guide the speech synthesis.

While the technology is amazing, I am a bit bothered by all these picture and video modifying algorithms. The issue is that we can't know what's real any more. It used to be if you saw a video or a photo depicting an event you could be pretty sure that what you're looking at actually happened. Now, if you see a video of a prominent politician saying something awful in your twitter timeline (or whatever), they may have actually never said anything remotely close. It could be a completely fictional video that looks perfectly realistic[0], made by some teen in Macedonia.[1] I realize photography and video have always been used to trick people into thinking things that aren't true, but this technology enables nuclear-grade deception. I am wondering: is there a use-case for such an algorithm that is practical and good for the world? PS: I know an eye-rolling algo is quite innocuous but I've had this thought on my mind about these in general and needed to air it out. [0] ▶ this video [1] link