Fake videos of real people — and how to spot them | Supasorn Suwajanakorn

Look at these images.

Now, tell me which Obama here is real.

(Video) Barack Obama: To help familiesrefinance their homes, to invest in thingslike high-tech manufacturing, clean energy and the infrastructurethat creates good new jobs.

Supasorn Suwajanakorn: Anyone? The answer is none of them.

(Laughter) None of these is actually real.

So let me tell you how we got here.

My inspiration for this work was a project meant to preserve our lastchance for learning about the Holocaust from the survivors.

It's called New Dimensions in Testimony, and it allows you to haveinteractive conversations with a hologramof a real Holocaust survivor.

(Video) Man: How did yousurvive the Holocaust? (Video) Hologram: How did I survive? I survived, I believe, because providence watched over me.

SS: Turns out these answerswere prerecorded in a studio.

Yet the effect is astounding.

You feel so connected to his storyand to him as a person.

I think there's something specialabout human interaction that makes it much more profound and personal than what books or lecturesor movies could ever teach us.

So I saw this and began to wonder, can we create a modellike this for anyone? A model that looks, talksand acts just like them? So I set out to see if this could be done and eventually came up with a new solution that can build a model of a personusing nothing but these: existing photos and videos of a person.

If you can leveragethis kind of passive information, just photos and video that are out there, that's the key to scaling to anyone.

By the way, here's Richard Feynman, who in addition to beinga Nobel Prize winner in physics was also known as a legendary teacher.

Wouldn't it be greatif we could bring him back to give his lecturesand inspire millions of kids, perhaps not just in Englishbut in any language? Or if you could ask our grandparentsfor advice and hear those comforting words even if they're no longer with us? Or maybe using this tool, book authors, alive or not, could read aloud all of their booksfor anyone interested.

The creative possibilitieshere are endless, and to me, that's very exciting.

And here's how it's working so far.

First, we introduce a new technique that can reconstruct a high-detailed3D face model from any image without ever 3D-scanning the person.

And here's the same output modelfrom different views.

This also works on videos, by running the same algorithmon each video frame and generating a moving 3D model.

And here's the sameoutput model from different angles.

It turns out this problemis very challenging, but the key trickis that we are going to analyze a large photo collectionof the person beforehand.

For George W.

Bush, we can just search on Google, and from that, we are ableto build an average model, an iterative, refined modelto recover the expression in fine details, like creases and wrinkles.

What's fascinating about this is that the photo collectioncan come from your typical photos.

It doesn't really matterwhat expression you're making or where you took those photos.

What matters isthat there are a lot of them.

And we are still missing color here, so next, we developa new blending technique that improves upona single averaging method and produces sharpfacial textures and colors.

And this can be done for any expression.

Now we have a controlof a model of a person, and the way it's controlled nowis by a sequence of static photos.

Notice how the wrinkles come and go, depending on the expression.

We can also use a videoto drive the model.

(Video) Daniel Craig: Right, but somehow, we've managed to attractsome more amazing people.

SS: And here's another fun demo.

So what you see hereare controllable models of people I builtfrom their internet photos.

Now, if you transferthe motion from the input video, we can actually drive the entire party.

George W.

Bush:It's a difficult bill to pass, because there's a lot of moving parts, and the legislative processes can be ugly.

(Applause) SS: So coming back a little bit, our ultimate goal, rather, is to capture their mannerisms or the unique way eachof these people talks and smiles.

So to do that, can weactually teach the computer to imitate the way someone talks by only showing itvideo footage of the person? And what I did exactly was, I let a computer watch 14 hours of pure Barack Obamagiving addresses.

And here's what we can producegiven only his audio.

(Video) BO: The results are clear.

America's businesses have created14.

5 million new jobs over 75 straight months.

SS: So what's being synthesized hereis only the mouth region, and here's how we do it.

Our pipeline uses a neural network to convert and input audiointo these mouth points.

(Video) BO: We get it through our jobor through Medicare or Medicaid.

SS: Then we synthesize the texture, enhance details and teeth, and blend it into the headand background from a source video.

(Video) BO: Women can get free checkups, and you can't get charged morejust for being a woman.

Young people can stayon a parent's plan until they turn 26.

SS: I think these resultsseem very realistic and intriguing, but at the same timefrightening, even to me.

Our goal was to build an accurate modelof a person, not to misrepresent them.

But one thing that concerns meis its potential for misuse.

People have been thinkingabout this problem for a long time, since the days when Photoshopfirst hit the market.

As a researcher, I'm also workingon countermeasure technology, and I'm part of an ongoingeffort at AI Foundation, which uses a combinationof machine learning and human moderators to detect fake images and videos, fighting against my own work.

And one of the tools we plan to releaseis called Reality Defender, which is a web-browser plug-inthat can flag potentially fake content automatically, right in the browser.

(Applause) Despite all this, though, fake videos could do a lot of damage, even before anyone has a chance to verify, so it's very importantthat we make everyone aware of what's currently possible so we can have the right assumptionand be critical about what we see.

There's still a long way to go beforewe can fully model individual people and before we can ensurethe safety of this technology.

But I'm excited and hopeful, because if we use it right and carefully, this tool can allow any individual'spositive impact on the world to be massively scaled and really help shape our futurethe way we want it to be.

Thank you.

(Applause).