Video Measurement Is Broken. Here’s How to Fix It

This story is part of Contently’s Accountable Content Series, a collection of articles, webinars, case studies, and events we’ve designed to help marketers deliver measurable brand impact and business outcomes with content. To see more content in this series, click here.

In Arthur Conan Doyle’s novel The Sign of Four, Sherlock Holmes says one of his most iconic witticisms, “I never guess. It is a shocking habit—destructive to the logical faculty.” Luckily for Holmes, he never worked in marketing. The famous detective may have had brilliant deductive powers, but even he would’ve struggled to make sense of today’s digital video landscape. (Although he definitely would’ve been a YouTube star.)

With a blog post, you know what to look for: readers. If you’re a little more advanced, maybe you focus on the time users spend on a page. But with video, the metrics are crude; every platform has a different definition for how much time counts as a view. That inconsistency forces publishers to guess what’s working, what’s not, and what’s just flat-out illogical.

That’s a problem because, as we’re reminded every other week, digital video is not only the future of marketing, but the internet at large. Cisco estimates that video “will be 82 percent of all consumer internet traffic by 2020.” Facebook VP Nicola Mendelsohn predicted that the social network would be “all video” by 2021. And in 2016, the Interactive Advertising Bureau (IAB) found that digital video ad spend increased by 85 percent over the previous two years.

As everyone rides the video boom, marketers have used bigger ad budgets to produce more experimental work. Geico, for instance, won a Grand Prix at the Cannes Lions festival in 2015 for a five-second pre-roll ad in which the narrator says, “You can’t skip this ad, because it’s already over.” Other brands have broken the third wall and included calls-to-action in their pre-roll. The German automaker Opel even used the “Skip” button as part of one clip, showing a car driving over it.

But no matter how creative brands get, ambiguous measurement tactics may be preventing the reality from matching the hype.

“In TV, we know how brand advertising works. You see a commercial. It has music behind it. There’s a little mini-story behind it. It’s always fifteen or thirty seconds long. It always takes up your entire TV screen,” said Jonah Goodhart, co-founder and CEO of Moat, an analytics company aiming to help people evaluate digital video performance. “But how does that work in digital? Is it a static banner ad? A full-page takeover? Does it matter if you can see or hear it all?”

They’re all important questions. Now we’re finally starting to get some answers.

Get a clue

When Moat was founded in 2010, viewability was just starting to become a big issue. Since then, the growth of video platforms like YouTube and social networks like Facebook, Instagram, and Snapchat have turned the issue into a digital free-for-all.

For all content besides ads, YouTube counts a view when a video plays for about 30 seconds. On Facebook and Instagram, it’s three seconds. Snapchat starts the clock as soon as something loads.

For ads, the equation is different. In 2014, the Media Rating Council (MRC) instituted a formal standard for what qualifies as a view: At least 50 percent of the video ad must be on someone’s screen for a minimum of two consecutive seconds. Many platforms have adopted this definition for their advertising businesses. (It’s worth noting, though, that Facebook still sells ads based on two options: impressions or 10-second views.)

Despite the universal threshold, only 66 percent of video ads, excluding YouTube, were viewable last year, per Google’s 2016 “State of Play” report. Additionally, in an email to The Wall Street Journal, MRC CEO George Ivie clarified that the two-second views “represent a minimum standard for the ‘opportunity to see’ the ad, they’re not about whether the ad was in fact actually seen or whether the ad’s message was actually received by the user.”

Autoplay and video player sizes complicate matters even more. For example, Facebook videos start playing automatically in a small window without sound when visible in a News Feed. Users have to click on a YouTube link to start watching, but the video player is larger. On Snapchat, the video may autoplay, but it’s the only thing in view.

“You would never design the system the way it is now if you could start from scratch.”

“I think it makes advertisers’ lives much, much easier if there is a standard,” said Tony Haile, founder and former CEO of the analytics firm Chartbeat. “However, in an ideal world, the advertisers should pick the metrics that best match the goals of their campaign… If you’re going for a two-second view but the first two seconds of a video ad make no mention of your brand whatsoever, you can have a highly successful campaign by the metrics, which does absolutely nothing for you.”

But what if a universal metric could do all that?

In late 2016, Moat unveiled the Moat Video Score, which incorporates how long someone watches a video, how long someone listens to it, and how much of the clip is visible on the screen. The metric consolidates the information into a rating between zero and 100, giving publishers and advertisers an easy way to measure complex insights. It also helps marketers translate results from different platforms. If a user watches a 30-second YouTube ad all the way through, in full-screen mode, and with the sound on, that gets scored a 100. But if someone watches that same ad on Facebook for 15 seconds and doesn’t turn on the sound, it would be less than 25, depending on how much real estate the video player took up on screen.

“What we try to do is connect our inputs to outputs, to business outcomes, so we can understand that when somebody is exposed to a certain type of ad on a certain site for a higher duration, that leads to higher ad recall,” Goodhart said. “We’ve connected our data with offline sales, as an example, for some of the CPG companies in the world so that we can understand what metrics are actually driving brand sales.”

Addressing all of these factors at once should change the conversation for the better. But overhauling the status quo won’t happen immediately, because not everyone is going to be thrilled with Moat’s movement.

Case closed?

For years, the video market was a cash factory marred by fraudulent traffic and ineffective programmatic ad exchanges. Money keeps pouring into it, but a 2016 eMarketer report showed that more than 70 percent of brand marketers are still concerned about viewability, click fraud, and bot traffic.

According to Jason Kint, CEO of the media trade organization Digital Content Next, a deluge of ad technology made it harder for advertisers to see where their investments were going. “You would never design the system the way it is now if you could start from scratch,” he said. “It was all built with the noble pursuit of more automation and efficiency. But we’ve got … a lot of opacity, and the mystery is causing a lot of people … to bring home great margins at the expense of trust in the marketplace.”

Perhaps the most high-profile case of misinformed measurement occurred last fall, when Facebook revealed it had been showing inflated stats for its “average duration of video viewed” metric to publishers and advertisers for two years. The mistake didn’t impact costs, but it did undermine some of the hype surrounding video. A setback like that gives publishers and advertisers a reason to question the accuracy of other data that gauges video efficacy.

With the current system, those trust issues will likely never go away. However, Moat and some other organizations like Digital Content Next believe there is a solution that could propel video forward: attention time.

“There is a lot of excitement to get more into an ecosystem where time is the unit of transaction,” Goodhart said.

Such a move would let buyers and sellers capture the true value of video. Brands would only pay for the total duration of how long users watch their ads. Publishers and platforms that sell ad space wouldn’t have to peddle cheap impressions. Plus, there’d be no more debate on what should qualify as a view, regardless of whether it’s zero seconds, two seconds, or some other yardstick.

“The question is whether we’re trying to figure out effectiveness or we’re trying to figure out existence,” Goodhart added.

Some publishers have started inching in the direction of effectiveness. The Economist and the Financial Times now sell ads based on attention. And a few years ago, Digital Content Next ran a small study of media companies including the likes of Condé Nast, ESPN, and The New York Times, which found that 80 percent were “interested in transacting on the basis of time.”

While the major players involved may not kill off the impression tomorrow, at least there’s some evidence we’re headed toward a healthier video ecosystem. After its metrics controversy, Facebook even agreed to let the MRC audit its data reporting.

“That doesn’t mean they are going to standardize the same metrics as other people, but it shows a certain amount of movement toward common ground,” Haile said. “The thing with all of this is we’re still in the early days. People are trying to work out what works. It’ll be interesting to watch.”