How Does the ESRB Rate Video Games?

(pulsating music)(keyboard keys clicking) – [Announcer] This documentaryis rating N for Noclip.

(gentle percussive music) – Over the past 25 years, video games have evolved in almost every conceivable way, giving birth to new genres, new ways of playing, and experiences as variedas the people who play them.

But there's one thing eachof these games has in common regardless of the type of game, who it was aimed at or theconsole it came out on.

Each of these games has passedthrough the hallowed halls of the Entertainment SoftwareRating Board, the ESRB.

The ESRB is a self-regulatory organization that is responsible forassigning age and content ratings to every video game released in America.

And while countless games have been through the doors of the ESRB, the organization has neveropened its doors to the media.

No journalist, let alone a camera crew, has ever been allowed inside.

But that is about to change.

In celebration of their 25th anniversary, the ESRB extended a specialinvitation to Noclip.

So we spent two days at their offices, chatting to the folks who work there, learning about the work they do, and sitting down tointerview a select few.

The ESRB by the very nature of its work has always had an airof secrecy around it.

And if our experienceinterviewing developers has taught us anything, it's that in a vacuum of understanding, misconceptions run riot.

So we aimed to use this opportunity to clarify what the ESRB does, to uncover their processes, their motivations, and how the work they do impacts gamers and parents every single day.

Our mission today is to uncover how exactly the ESRB rates video games, to answer the question of who it is they ultimately answer to, and to see whether or not thisrelatively small organization is able to keep up to speed with the incredibly fastindustry they preside over.

(soft pensive music) – So back in the day beforethe ESRB, games were evolving.

Resolution was improving, audience was expanding, there were games like MortalCombat that were coming out that were graphic for in those days, Doom, and there was noconsistent standard.

There were some attempts at ratings but there were no standards, and the government, particularly two legislators, two senators, started to really question whether the industrywas acting responsibly and if the industry didn'tcreate some standard, they threatened that the government would.

So they held a series of hearings.

The industry quickly assembled itself and created a trade associationand created the ESRB, but up until that point, no, there was really no consistencyand no collaboration and no even will to collaborate on it.

So, '94, we started assigning ratings.

– [Interviewer] When didthose initial ratings first get established? – Well, we had film ratingsbut I think they realized that they wanted tocreate their own system and I'm sure there was a lot of debate.

Having not been around in those days, I'm sure there was a lotof debate about which ages, which thresholds, and we changedthe ratings over the years.

We started out with a K to A, which is now our Everyone rating.

We didn't have an Everyone 10+.

We just had a K to A andthen it went up to Teen.

Now we have a rating in between.

We were the first rating system to have both age ratingcategories and content descriptors because our research at the time indicated that parents really wanted to have both.

Age ratings are greatbut parents wanna know why you got a particular age rating because parents have different opinions about different types of content.

Some parents may be okaywith violent content but not sex or language.

Other parents may be morecomfortable with sex or language but not violence, so it's really important to provide both parts and nowof course we have a third part which is Interactive Elementsbut that's much later.

– [Interviewer] Beforewe dive in any further, I asked Pat to clarify afew things from the outset.

First of all, the ESRB areoften conflated with the ESA, the Entertainment SoftwareAssociation that runs E3 and lobbies government onbehalf of many major publishers.

So I asked Pat to clarify the relationship between the two organizationsand to give us the answer to that age-old question, can the games industryeffectively self-regulate itself? – Technically we are part of the ESA.

We operate independently, obviously.

They're in D.

C.

, we're in New York.

We have a completely different mission.

They're very member-driven.

We are taking industry-adoptedguidelines and enforcing them and we're really the face of the industry for informing parents and for making sure that we're acting in a responsible way and it's just a verydifferent sort of mission, but the board, our boardof directors is the same as the board that runs ESA.

They put their goodcitizen hats on with me and do the right thing.

It works 'cause they're bought into it.

They believe that as an industry, self-regulation is the best approach.

My attitude is self-regulation, the proof is in the pudding.

We've been around for 25 years.

Parents trust us, they use us.

What more can we do toprove that we're serious about what we do and we're good at it? So, are the ESRB good at what they do? It's a good question andto find out the answer, we're taking a look atthree core responsibilities that the organization undertakes.

First we're gonna lookat how games are rated.

Then we're going to explore how the ESRB enforcesadvertising standards in trailers, commercials, and print ads.

And finally, we're going totake a look at how, if at all, the organization hasevolved with the times, whether or not theyhave effectively adapted to new challenges suchas digital distribution, indie self-publishers, and thedesire that many gamers have that they step in onthe issue of loot boxes.

Our first stop is with the ratings team, but before we learn howexactly they rate games, there was one question I had in the back of my mind all day.

What type of person works at the ESRB? – So I was actually outof work for a little bit.

My first day of unemploymentwas September 10th, 2001, in New York City, and Idecided I would start looking for work the next day, and of course, that was September 11th so that was exactly 18 years ago today.

And it was really hardto find work in the city.

Nobody was hiring, itwas the shock at first and then after that, there was questions of where people were evengonna have their offices and things like that.

And my downstairs neighboractually worked at ESRB and I would sort of jokingly say, “When are you gonna getme that video game job?” which kinda kept ongoing and going and going and so I started to kind ofexpand the sort of things I was looking for outsideof software development and eventually he got calledinto the police academy.

So he said, “Hey, there'sthis job opening up.

” So then I started August 2002.

– My story is also relatedto another employee who actually is stillhere, Shayne Spaulding.

He had recently startedworking at the ESRB.

I had met him a couple timesbut he knew I was a big gamer.

Gaming was pretty muchmy life at that time and it still is andwill probably always be.

He brought in my resume andI came to the job interview with a paper that I hadwritten for a class in college.

– You should've seen this thing.

– (laughs) I wrote it for anElectronics Media Policy class.

So I was in an electronicsmedia program, but at the time, there weren't really a lot of specialized video game programs but I tried to steer it towardsgaming as much as I could so I figured in this course, I would kinda deviate fromwhat the assignment was a little bit and writea paper about the ESRB.

– [Interviewer] You're kidding.

– No.

– Yeah.

– So I wrote a big paper about the ESRB.

– The most well-prepared candidate you could possibly imagine.

He's like, here's mycollege paper on you guys.

– Yep, and I brought copies for everyone.

I kinda handed it out and I was, I mean, I was a little starstruckat the time walking around.

I knew Pat Vance's name and I was like, oh my god, do I get to meetPat Vance, this is super cool.

– He knew, yeah, he knew everybody's name.

He was looking down the hallway and things trying to spot— You were an ESRB fanboy.

– Yeah, kind of, yeah, yeah.

– Kinda, yeah.

(laughs) – [Rocco] So yeah, I camein here with that stuff and I think you had said in the past that you were close tohiring somebody else.

– Well, I mean, you hadthe book report done, so.

– I did, and I think– – That pushed you over the top.

– I dropped, it thudded on everyone's desk with a resounding, and I, here I am.

– Yeah, no, I definitely, we've talked, I will wake up in coldsweats thinking oops if I didn't hire, like, that's the nightmare, I accidentally didn't hire Rocco, so.

– I do like that I can still be the source of your nightmares.

– Yeah, yeah, yeah, no, that's true, that is true.

– Yeah.

(interviewer laughing) – [Interviewer] Video gamesaren't the only form of media that contain age ratings, but the challenge of applying thoseratings is surely unique.

A movie can be watchedover a couple of hours.

A music album, even less time.

But to apply the same level of scrutiny to an interactive medium wheregames can contain dozens, even hundreds of hours worthof bespoke, player-led content seems like an impossible task.

During our time at the ESRBwe were given one caveat.

We weren't allowed to film the raters.

They work under anonymityto protect themselves from publisher influence.

The raters come from all walks of life but are chosen based on a certain level of video game literacy andexperience with children.

They work in a private room, viewing hours upon hours of video game footage every day.

But that's just one piece of this process, so we asked Rocco andBill to take us through the procedure of rating a game.

(pensive music) – [Rocco] The first componentis the online submission form, so that's basically a webform that you fill out where you tell us about yourgame, you tell us, you know, you give us some generalinformation about the game, how it works, what you do, and then you tell us aboutyour pertinent content.

That's our important ESRB-defined term.

Pertinent content meansthe most extreme content in your game in its relevant context.

So we don't wanna see just your violence.

We don't wanna see just your sex.

We wanna see the build-up, the lead-up.

We wanna see what happensafter a head is cut off.

Why is it getting cut off? What happens, what's the aftermath? – Yeah, we don't want anextreme content montage where it's just decapitation, decapitation, decapitation.

So it should show us thelead-up, like he was saying, and all that.

– Yeah, it can bias people to have them see somethingthat's more extreme.

It doesn't really get intothe context of the game.

Most games have, you know, violence is a primary mechanic in a lot of video gamesbut there are story.

There's all sorts of other stuff that, it's not the only thing that you're doing.

So you fill out, they're essentially long-form essay questionswhere you tell us about the violence, the sexuality, the gambling, thelanguage, the crude humor, the different contentcategories that we care about and that parents care aboutand you basically write the most extreme instances and then you, at the same time, prepare a video.

You upload an HD video generally in your game'starget output resolution to our secure FTP andthat enumerates the things that you're talking about on the form.

So that gives us the examples, it gives us the context.

You show us low-levelplay, mid-level play, cut-scenes where appropriate.

And we try to put it infront of our radars in a way that is as similar to whatan end user would experience without actually having acontroller in your hand.

So you upload that to us, you agree to our terms and conditions which says what you canand can't do with a rating, you pay a fee, then thatcomes in to my staff.

My staff goes through everything, works with the publishers to make sure that it's appropriate, that it's how we would expect it, that it does a service to your game, that you're not sending usan extreme content montage, as we said, we polish it up and then it goes in front of our raters.

– Yeah, and as long as everything's good, we have all the materialsthat we need, at that point, we'll set it up in front ofraters who will review it.

There's at least threeraters, they're anonymous.

There's a big wall betweenthem and publishers.

They're not allowed to speak to publishers or talk to press or anything like that.

– [Interviewer] We'renot allowed film them.

– Right, you're not allowed to film them.

So they're allowed to do their job in an unbiased and unpressured way.

They can just react to content.

So they'll go into the rater room.

They'll review the footage and after that, they'll come into thisoffice and we'll have, one of the raters isdesignated as the foreman and they'll come in and they'lldiscuss the rater feedback.

The foreman will alsodraft the rating summary and James Chang, directorof rating assignments, will also come in and he'll do what's called a parody analysis where we'll look at similarrated products in the past and make sure thateverything is consistent with precedent and all that.

After that meeting, the rating's finalized.

The publisher gets it.

If they're happy with it, which, most of the time, they know what they're gonna get.

Most of the time they have an expectation and it's aligned with that, that's sorta the end of the process.

If they're not happy with the rating or they have questions about it, we're happy to tell them whyit received the rating it got.

If they want to edit the product to get a different rating, they can do so, but that starts thewhole process over again.

And we don't tell them what to do either which is a critical point.

We just inform them why itreceived the rating it got.

We don't say, if theysay, “We got a Mature, “we wanted a Teen, ” we don't say, “Well, do this, do this, do that.

” We say, “Here's whattriggered the Mature, ” and go through that and then they can take whatever creative choices they wanna make to achieve their goals.

– I would say they averagebetween 30 to 45 minutes but they can go as longas four to five hours, sometimes even longer thanthat in very, very rare and isolated instanceswhich is very tough for us 'cause that can be longerthan a business day and sometimes we have to splitthe session over two days and we definitely wanna avoid that.

The majority of stuff can be accomplished in under 45 minutes, we think.

– Dialogue-heavy products can be tricky because then you're lookingfor sorta specific lines.

The core mechanic repeats alot throughout the gameplay so if it's just talkingabout it's a fighting game, the fighting mechanicrepeats over and over again so that's a little, I think, probably easier.

– Yeah, like, a bigger open-world game orsomething is gonna be harder.

It's gonna take more time and generally the videos will be longer, the submission will be more dense than something like Tetris.

– [Interviewer] Theprocess of rating a game usually takes between fiveand seven business days, but that's assumingeverything is disclosed.

So I asked the guys what happens when pertinent content isn't disclosed, when the ESRB finds inconsistencies between the game footageand the paperwork.

– There's two main time periods.

There's during theprocess and post-process.

So post-process is much more serious.

We discover somethingwhen it's been released but there's in-process violations.

– Yeah, so that's part of the compliance.

Part of my job is dealingwith those types of issues.

So we have what's called the clarification which is exactly what it sounds like.

Hey, what's up with this, explain it.

And then if it's aparticularly egregious omission either on your video or on your form, we have our enforcement system.

We can give an in-process enforcement.

We call them Class C violations.

Those are the more serious of the in-process things that we can do and you have two morningpoints built into that so the first two don't come with a fine but beyond that you can get fined for more egregiousissues of non-disclosure when your games are in herebut that's how we do it.

The majority of stuff, when we have an issue during the rating process, it's a clarification.

It's just an email.

It's like, explain thismechanic, explain this issue.

Why didn't you talk aboutthis, blah, blah, blah, and generally that's the end of it.

But if it's a more seriousthing that they missed, and it's not people tryingto hide stuff from us.

It's usually new people, peoplethat just miss something, people where it's buried in their game, they didn't think about it andthey didn't think they needed to disclose it.

– Software's complicated.

It moves quickly, thingsare getting developed and we have never seen an example of someone trying to hide something.

There's really nobenefit to them to do it.

We don't think anybody'strying to game the system.

It's generally, when we findstuff, it's smaller in nature.

No one misses disclosing the violence in a fighting game, for example.

It's generally, there's a character page and someone says they like wine and that's the thing that they miss.

So post-release would beif we discovered something that was on the shelf.

Those are more serious violations.

It's essentially the samething where they get a letter and they have to update the packaging and update the product, but similarly, there's points andfines associated with it and they can go up to amillion dollars, actually.

(crowd chanting) – [Interviewer] These dayswe can capture gameplay and transmit large videofiles with relative ease but anyone who's workedonline or in games knows that this is a relativelyrecent indulgence.

So how did the ESRB tackle the job of reviewing game footageback in the days of VHS? – Yes, yes, we weregetting tapes and faxes.

Faxing was part of the system.

– You can still fax us though.

– Well, you can, we don'twant them to fax us.

I mean, there is a machine, but– – We get the occasional fax.

– It's like, you know.

– [Interviewer] Please don't fax us.

– Yes, don't fax.

(interviewer laughing) So yeah, so they're faxing, VHS tapes.

We had situations wherethe video game would end and Seinfeld would kick in.

(interviewer laughing) – You just grab whatever tape you had lying around, right?- Yeah, people would take their old tapes.

– We were on VHS tapes way too long, until 2008, I wanna say, we were on VHS tapes.

– Shh, don't let them know.

– Yeah, yeah, sorry.

– Yeah, no, we were on, we were sort of– – That was one of mainstated goal when I came in.

I'm like, I'm dragging this organization kicking and screaming off of VHS.

– Yeah, that was good.

– Did indeed.

– [Bill] It's hard tofind a VHS player in 2008.

– [Rocco] They're expensive.

– [Bill] Yeah, exactly.

– [Rocco] DVD for a while.

– DVD for a while.

– We are all digital now.

We've been all digital for a while.

– You can FTP, the works.

– Yes, upload your HD video please.

– Which is super, one of the, people don't thinkabout how inefficient it is.

I mean, the VHS, whenyou talk about a scene, you've gotta rewind it.

– Oh, god.

– You're just sitting there– – Nightmares.

– Watching, yeah, I mean– – DVD was a pain with that too but it was quicker.

– It was better, it was better.

– But we found these DVD players that could goup to 100X for fast-forward.

– Wow.

– So yeah, we got these Prosumer playersand we put them everywhere.

– Yeah, and then, like, 80% of them died within a year of each other 'cause we just ground them to a paste.

Yeah, when you take aboutchanges, the industry itself, the game was done, it goes in a box.

There's no such thingas patches and whatever.

Now that's the beginning, is the retail process.

– The first thing you do is you put your game in and you download an 11-gig data patch which, 15 years ago, you got your cartridge and that cartridge was never patched.

They might do another manufacturing run to fix some bugs or somethingbut you would never know, and we don't just issue the ratings.

We also, when the games come out, make sure that those ratings are accurate.

So that's another partof what I oversee here, is our post-release testing process where we try to test allof the new long-form games that come out or allthe new long-form games that people are caring about, put at least four hours, if not more, of testing intoevery game that comes out just to make sure that what they told us during the submission process is accurate.

– And that's a great point because we also recentlyupdated our system to help identify products that have long content plans post-release.

Special editions often haveseason pass and things like that so that helps flag it for us to know that not only do we needto do that initial big test but we'll probably need tokeep checking it over the years to make sure— Games as a service is such a big thing thatyou could buy a game and then play it five years later and it'll be totally different, so we need to keep looking at it.

(pensive music) – The rating team is just one part of the enforcementresponsibilities that the ESRB has.

Directly across the hall fromthem are the team responsible for making sure that everypiece of video game marketing that reaches a consumer abidesby a set of ESRB guidelines.

Dave Gossett came from the music industry and was used to managingrappers and rappers' managers during the golden age of hip-hop.

These days he's the director of ARC, ESRB's Advertising Review Council.

They're the folksresponsible for making sure that every trailer, TVcommercial, billboard, and advertising have appropriate content and are targeting the right demographic.

And while you may not haveheard Dave's name before, you've almost certainly heard his voice.

– I've been at the ESRB 19 years.

This will be my 20th year in December.

– [Interviewer] Congratulations.

– Thank you.

– [Interviewer] Are yougetting anything for it? – Am I getting anythingfor, like a gold watch or something like that?- Cake or yeah, a gold watch.

– I'm sure there'll be something pleasant.

– [Interviewer] Somebody told me also you have quite a famous voice.

– Absolutely, I consider myselfthe voice of the industry (both laughing) for those that know.

Rated M for Mature.

Publishers can use whatevervoice actors they want to to create the voiceovers that are required in our television spots butthere would be occasions when they didn't have avoice actor and they said, “Well, do you guysprovide the voiceovers?” And at that point we didn't, so myself and a young mannamed Blake Christiana, we took a day in the studioand he engineered the session and I recorded all of the voiceovers.

So we actually have aprofessional voice actor who provided voiceovers for us in English and otherlanguages and then my voice and we actually took my voice off after we had the professionalvoice actor do it but all the publisherscalled back and was like, “Where's the other voice?” (laughs) And so it's nice to be ina bar doing NBA playoffs and commercial for basketball game and you hear, “Rated E forEveryone, ” that's your voice.

Rated E for Everyone.

– [Interviewer] Can you do an M? – Rated M for Mature.

(interviewer laughing) Rated E for Everyone, rated T for Teen.

We're responsible for ensuring that video game advertisingis appropriately labeled with our rating informationand equally as important, if not more important, appropriately targeted to the audience that it's meant for.

– [Interviewer] So how do you, I guess, go about what you'repolicing is the wrong term but enforcing that or guiding people? Are there specific guidelines?- I think policing, enforcing, and guiding arekind of all the right terms.

On our website we have a publisher section and publishers have accessto what we call an ARC manual and inside that ARC manualare all of our requirements for marketing and advertising guidelines, for rating icon placement andcontent descriptor font size, kind of all the technical stuff, and then we have a sectioncalled Principles and Guidelines for Responsible Marketingand Advertising Practices and that's where we kindaget into the discussion about content and what isand isn't acceptable in ads and its various media or mediums that the advertising encompasses.

There's television, magazines, a lot of social media now.

Out-of-home advertising is huge, your billboards and your posters.

So I'll do Times Square or L.

A.

LIVE.

They get really big.

They can be just normal-sized poster or billboard somewhere on the highway but some of these budgets are pretty big and you'll have a huge AAAtitle where you've got– – [Interviewer] So E3, the– – [Dave] E3, I look at all that stuff before E3 happens, absolutely.

– [Interviewer] What about thestuff on the Hotel Figueroa? – Hotel Figueroa, I know the requirements like the back of my hand.

I know exactly what theyare, what we require, what the Hotel Figueroa requirements are, and that's always one ofthe first things we look at when we show up to the convention center is we look at the Hotel Figueroa and see how huge the rating icon is.

I mostly focus on content and the content is usuallyin television spots, in trailers, in general audience videos or paid advertising videos for use online so I'm constantly looking at the content that publishers provide us and kinda going through frame by frame and saying, okay, for television, we apply our guidelines moststringently so this scene here, this depiction of thisgraphic or violent act isn't compliant with our guidelines so you need to edit or remove that scene.

And that's really what Ido on a day to day basis all day long for all thepublishers who submit.

I'm very sensitive totelevision commercials because that's a passive audience so you're just sittingat home watching a game or some show and then a videogame advertisement comes on.

That's kind of wherewe apply our guidelines most stringently so I'malways on the edge of my seat when a television spot comeson 'cause it's a huge screen and I have good screens in my office to look at this stuff on but you're always on the edge of your seat hoping you didn't miss anything.

I look at those commercialsvery differently, I think, than everyoneelse that looks at them.

(smooth music) Just because there's graphicviolence in your video game doesn't mean that we'll allow you to display graphicviolence in your marketing and advertising materials.

We have principles and guidelines and one of our guidelines is violence and then under that violenceguideline we have examples.

No characters being shot, for example, or no blood.

They're definitely suitablefor a general audience and I also take pride in understanding what the publishers are going for and conveying to the consumerexactly what's in the game.

So it's kind of a fineline you have to walk where we're applying ourprinciples and guidelines and making sure the ad iscompliant with our guidelines but also making surethat the consumer knows the type of content that's in the game.

– [Interviewer] Is there ever any issue where TV spot again, in particular, it's communicating asense of fear or dread or something like that where something is, 'cause you could have, things can be scary without being violent.

Is that ever a part of it? – Absolutely, so we have something that we call the overall tone of a spot.

I'll look at it once as aconsumer would look at it and then I may break it downinto smaller components, scenes, but I'm alsogoing for an overall feel.

We've got a lot of first-person shooters where there are televisionspots made for those so we have guidelines aboutan excessive amount of gunplay and our job is very subjective.

The good thing about us beinghere for such a long time is we have a lot of historywith the publishers, with the content, withthe type of content, with what our industry does, so I feel like I have a goodbarometer for what's excessive and what's graphic and what's gratuitous.

So yeah, overall toneis definitely something that we pay attention toas well as staying abreast of what's happening in the world today.

– [Interviewer] Is theremuch back and forth between you and thestakeholders on the other side? I imagine there is.

– Abso-freaking-lutely, one ofthe things I pride myself in is that I deliver a lot ofbad news on a daily basis.

Like, you have to takeout this, this, and this from your ad, your video, your trailer.

We as a department encouragethe back and forth.

So I know we're the whipping board for a lot of fans out thereand I know a lot of people and one thing I enjoy about doing this, and I think I mentionedthis to you earlier, that we're not gray-hairedold men in suits just looking at this stuff.

We're a lot hipper thanpeople would think, I would like to say.

– [Interviewer] Doesmost of your job involve the sort of higher endof the rating scale? – It's funny, preparingfor this interview, I was looking at thepercentages of games rated and so about 60% of thegames rated by the ESRB are in between the E andE 10+ rating category, and so you've got 30%that are about Teen rated and you've only got 9% of thegames that are rated Mature but I spend a lot ofmy time in the 9% realm and that's because a lot of the titles that have the bigger marketingand advertising budgets are your AAA Mature rated titles.

But there are other verysuccessful Teen titles where I need to review the content as well and some of the publishers that create some of the most influentialor newsworthy content are some of the most compliantpublishers out there.

So people might look at thecontent they create and say, “Oh, these guys are out there, ” but when you look at howthey go about doing their art and handling their business, they're very responsible.

(pulsating music) – Today's video game industryis almost unrecognizable from how it appeared whenthe ESRB was first founded.

While once a handful ofconsole manufacturers and retailers were the solegatekeepers to the market, these days, digitaldistribution, free-to-play games, and app stores risk circumventingthe established protocols.

So how does the modest-sizedESRB stayed relevant in a world where millions of games are released every single year? Their answer is something called IARC, a semi-automated rating process that could work acrossinternational territories.

The hope was that some form of automation would help bring ratingsto the growing number of app stores floodingthe market with games.

But it's fair to say not everyone at ESRB was on board from the start.

– The business was changing.

With mobile coming on, with digital, the volume was gonna becontinuing to increase, the type of developers that were coming into the market were different.

There were indie developers, very little resources, and I felt that it was really important for us to try to create ascalable solution for ratings.

Could we take what we do, which is highly subjective when you're looking atcontent and break it down into zeroes and ones, multiple-choice questions? You'll hear from others herethat the initial response I got from my team was no, adamantly no.

– We were not huge fans immediately.

– She is very forward-thinkingand I think we were stuck in a process, we were at that point where we didn't really drink the Kool-Aid.

– [Bill] Yeah, I think we were scared.

It's a big volume— Our process worked.

– [Bill] Yeah, we had, if yousorta walled off what we had and just kept at that, it was great.

And the traditional process we think is– – Still great.

– It's a great process.

It's just not scalable, and to solve the scalability challenge, we needed to think alittle bit differently, and it's hard for us becausewe had been sort of conditioned to not think differently and sort of say, it's really important to getthis completely brutally down before it releases because a correction or a mistake at retail is bad.

– So we first tested it withconsole downloadable games that were not available in physical form and we continued to tweak it and the model was sort of a hybrid model where the developer paid a fee.

It wasn't as high as aphysical game rating fee, but paid a fee nonetheless andstill had to submit a video but that there was still a questionnaire that had an algorithm that wouldassign the original rating.

And then it was up to us to make sure that that rating was correct after the product gets released.

So it was very risky, andthen I took that system to other rating authoritiesaround the world and I said, “If weren't doingour best to broaden adoption “across all of these new devices, “all of us as rating authorities “were gonna be marginalized to death.

” So we had to go around theworld and basically talk to other game rating authorities and say, “Is there any way that you would consider “using a system like this? “And we'll program a separatealgorithm for your region “'cause cultural norms aren'tthe same around the world.

” – And that was one of the things that we wanted to make sure was understood with what we were trying to pitch with IARC in the beginning.

We weren't trying to exporta U.

S.

standard out there.

So certainly different regions, I know when one of the store fronts was testing the questionnaire, they said that there wasa bug in the German output because the nudity was so low and they said, “No, that's afeature, that's not a bug.

” And that's what's great about IARC.

We can ask one question and then generate all the different outcomesthat are regionally appropriate instead of saying, “Here's the U.

S.

standard.

“We're gonna export it everywhere.

” – So the developer onlyanswers the questionnaire once but they get the ratings in each region and they wouldn't have to pay for it and they wouldn't have to wait for it, whereas in the traditionalratings business, you have to submit toeach rating authority.

You have to pay a fee.

They each have their different timelines.

So we were eliminating allof that with this system and we built a dashboardfor rating authorities so that they could see every single rating that's coming through and test it and modify the rating if they needed to.

– I can click a buttonon my computer over there and within 15 minuteshave multiple billions of devices fixed and that's something that you don't have thecapacity to do in retail.

So sorta seeing that way forward and building that framework– – I think it's witchcraft.

– Yeah, it is– – But the fact that you can click a button— It's magic, it's dark magic, really, it's not, that's how it works.

– And you can be in, like, Africa, on your android phone, and the rating is up like that.

How does that, it's crazy.

– That's how we can getthere and make it work.

– As long as it's implementedwell by the storefront, it really does work like a charm.

It does exactly what weintended which is to control the rating that's displayed in the store, give rating authorities theability to customize by region, and to monitor the accuracy of the ratings that are being assigned and be able to modify them very quickly.

(pensive music) – [Interviewer] WithIARC, the ESRB developed a new mechanism inanticipation of future change, but what happens whenchange comes out of nowhere and they're forced to adapt on the fly? A recent example of this iswhen Limited Run Games attempted to release limited edition boxed copies of small indie titles.

These were digital games that had never had a physical edition and so hadn't been throughthe full rating process.

The cost required to get arating would've been so high that it then didn'treally make business sense to do the limitededition physical version.

When news broke around this, the ESRB were criticized for not having a mechanism inplace for these smaller teams.

So I asked Rocco and Bill howthey were reacting internally.

– So we built a newprocess around that too called the digital physical rating process which essentiallyleverages some of the work that's already been done throughthe digital rating system.

It still has to come throughthe traditional process but it's a little quicker, it's a little cheaper for them to do that.

– It's our cheapest rate.

– Yeah.

– And so basically youneed to have been out in a digital form, validdigital ESRB rating for 90 days, and then you're eligible for that and that's generally howmost of the limited runs, the strictly digital developer, all those types of games come out.

So those come through us pretty frequently and those guys are the onesthat need a lot of hand-holding 'cause they're generally smaller teams and they're not superfamiliar with the process.

And at the same time, a lot ofthose people are the coolest and we're just happy to hold their hands 'cause their games are, (Bill laughing) like, if your game is a digital game and it's successfulenough to get a physical, then you're generally pretty happy and excited and it's a cool thing, so a lot of them arehappy to come through us even though those submissionsdo need more polishing from time to time.

– We don't wanna block peopleout from the rating process and we try to beaccommodating where we can.

– We're pretty quick todeal with stuff like that.

The industry moves really, really, really fast and it's hard to stay ahead of it but I think we're pretty good at it.

– [Interviewer] Justlike it was 25 years ago, video games continue to evolve, and the ESRB is oftenpulled into conversations regarding the widerpolicing of the industry, most recently being mentionedas a possible solution to the growing wearinessthat many gamers have toward loot boxes.

In reaction to the controversy, the ESA, the Entertainment Software Association, has said they will beintroducing new policies on the disclosure of itemrarity and drop rates, but the ESRB's mission, toeducate and inform parents, didn't exactly overlap withthe desires of many gamers.

So I asked Pat what was theESRB's reaction to this issue? people were asking usto disclose loot boxes but we did research very quickly and realized that parentsdidn't know what a loot box was.

When we told them what a loot box was and we asked them whattheir concerns would be, by far their concern was spending.

Now, gamers may beconcerned about it looks and feels like gambling and why don't you, ESRB, call it gambling? It doesn't fit into the criteria of either simulated gambling or gambling, so that was not an avenuewe were gonna go down.

[Danny] If somewhere down the roadthere was some sort of regulation brought it on loot boxes, would that be something you'd.

.

.

does it fall under somebody else's remit? [Pat] Well take the drop rate issue.

The industry in the US has made a commitment that drop rates are going to be partof loot boxes going forward.

And the reality is that drop-rates aren't for parents Drop rates are for the gamer there's already a lot of information that gamers can utilize about loot boxes and then in terms of disclosures, it was really important for us to make sure that we were providinginformation that was helpful to parents and we had been using an interactive element called “digital purchases” in the mobile and digital world so what we did a year and a half ago was to start using the in-game purchases descriptor in physical as well as digital and mobile But we are continually evaluating we conduct research ever year with parents.

we want to understandwhat their concerns are So our messaging now is two-fold It's about making sure theyknow that there are ratings but that they can dosomething about it too with these parental controlsthat just keep evolving and keep getting better and better.

– [Interviewer] This was the first time the ESRB had let cameras in the door, and from talking to thefolks around the office, you get a good idea of why.

Their work is kind of a thankless job, the type of place thatonly gets talked about when it comes into contactwith a hot-button issue.

The day to day here istaken very seriously, not only because of public scrutiny, but because of the variousmasters the ESRB has to serve.

For them, that means thegovernment who requires a high level of enforcementand parental awareness, developers who needgames to be rated quickly and accurately, and parentswho want to trust its source when it comes to choosinggames for their kids.

As a regulatory body, their performance is audited frequently but we don't really hearmuch about what they do on a day to day.

Perhaps that's why theylet us through the doors, to try and give us some insight into the difficult workthey do here every day.

There's a lot more that goes on here.

The ESRB Retail Council, for instance, is responsible for auditing stores and monitoring enforcement of sales.

In fact, during Pat's tenure as director, videos games haveachieved the highest level of retail enforcement of any industry, a statistic that means alot when certain people in congress are looking for an excuse to blame video games forreal-world atrocities.

And while the work may be difficult, it's clearly work they feel rewarded by.

So before I left I askedsome of them what it was about working at the ESRBthat makes them stick around.

– I've always wanted towork in the games industry and I did some game writingand stuff when I was younger but I didn't wanna pick a company and go work for that company.

Here I get to work with every company, literally every companyaround the entire world and that's awesome andthen we get to deal with, you know, your digital games.

We get to deal with apps.

We get to deal with every challenge.

We get to deal with gamesas a service, free-to-play, everything as it comesin, we're addressing it and so you're not doing thesame thing over and over again.

It's constantly being refreshedand it's constantly vibrant and I think that's whypeople stick around.

– We do believe in the mission.

We're really proud of what we've been able to accomplish here and I thinkwe do have new challenges and we keep on adapting to it.

So it's not just sortastamping the ratings over time.

– Yeah, the job isn't stagnant.

The job moves, the industry moves.

– It has changed so much and we've been part of that change.

We've been able to shape that change.

– For me, the public service component of it is really compelling.

It's not just that you'rein the video game business but you're actually providing a real valuable service to the public, and you're helping theindustry self-regulate and protect itself from regulations so I think the whole mix is a really unique and compelling mix.

At the end of the day, it's sort of all worth it.

(gentle pulsating music) – You wanna start? – (laughs) Well, I have this problem where I don't play a lot of games but I play a small number of games to maybe unhealthy obsession.

So I'm currently playing Battlefield V.

I just checked my playtime yesterday.

– Your slash played? (interviewer laughing)- Yeah, no, Warcraft killed me— I know.

– Because of that, yeah.

– Yeah, I know.

– Warcraft, the fact that– – Never look, never look.

– Yeah, never look.

What do you think, 'cause I had an idea.

(Rocco sighs) My guess was 500.

– Yeah, I was actually literally was gonna throw out 500 hours.

– 910.

– Jeez.

– Wow.

– In Battlefield V.

– That is too much.

– So that, Kerbal SpaceProgram, huge fan of that.

I played that with my kids a lot.

– (sighs) Let's see, todayalone, I have played Pokemon GO which I am continuingto play since launch.

Maybe I shouldn't be proud of that but I enjoy Pokemon GO.

I'm playing Doom, theoriginal Doom which just got, they re-released 1, II and 3 on everything so I've been playing that on Switch.

Hopefully they will fix that port 'cause there are some issues there but it's still Doom and it's still fun and it's a game I'vebeen playing since 1993.

I've been an id fanboypretty much my whole life so I love everything they've done.

I loved your Doom 2016 video.

It was one of my favoritethings that I've ever seen.

Thank you.

– Thank you.

– “Masters of Doom” is my favorite book.

– What else, Tetris, I've pretty much, I've played Tetris almostevery day since 1989, I wanna say.

– Really? – Yeah, I'm a bit of a Tetris guy.

I played through Blazing Chrome which is kind of likea Metal Slugs, Contra, if they had a baby, modern, kinda new.

Played through Cadenceof Hyrule which I loved.

– And it's great becausehe knows all these games pretty intimately and then ifthey become pretty successful, sometimes they'll wanna gointo the retail process.

Now they're gonna come to retail and he already is prettyfamiliar with them so if we get them in– – Yeah, a lot of timespeople are reaching out to me and I'm like, “Hey, Ibacked your Kickstarter.

“I know your game, I playedit in beta, what's up?” – Right.

(interviewer laughing) – Make sure you tell usabout this, 'cause I know.

(all laughing) – [Bill] Right, right.

(keyboard keys clicking).