How We Fail at Data and Analytics

Can you imagine anyone telling a race team that their data isn't important?

The most grueling motor race in the world, the 24 hours of Le Mans, happened earlier this month (and Audi won again). It's not just impressive in terms of the performance of the cars and the drivers, but more so in how the team uses data to bring it all together.

Sports like Le Mans are absolutely dominated by data. Every single person is obsessed with it, from the drivers, to the mechanics, the engineers, the team managers, the planners, the helpers... and even the fans.

Note: image via @24hoursoflemans

Unlike how things are measured in the media, every single measurement at Le Mans is targeted to monitor a specific need. And every piece of data is validated to make sure that everyone knows exactly what real-world outcome it is linked to.

And because they know what their data means, they can actively use it, in real-time, to make real-world decisions that have a huge impact on their ability to succeed.

Compare that to what we see in the media and marketing world. Media people generally do not understand the data their analytics is gathering. Neither do the tools that we use. Instead, we measure shallow things that don't really relate to anything in the real world. And because we don't understand it, we collect them as averages or totals for our reports.

Can you imagine a race team creating a report telling them what the 'average speed' or the 'average brake pedal pressure' is? That would be totally and utterly useless.

On a racetrack, you have fast sections and slow sections, curves and chicanes. And each part needs to be approached in a different way in order to maximize the result.

And it's exactly the same for brands and the media. Measuring things like pageviews, unique visitors, and average time-on-site is meaningless. It doesn't tell you anything about why some people convert or why someone are more loyal than others. It doesn't tell us anything about the real-world value of each article or product.

People might be spending more time (on average) looking at one product, but your actual sale might be higher for another. And yet, your analytics put the first product on the top of the list.

The result is a disconnect in which people are discouraged from looking at the data.

For instance, as American Journalism Review reported:

Editors at The Verge said they don't share detailed site metrics with writers because they want them to cover what's important, not just what's trendy.

Vox Media CEO Jim Bankoff said:

People have access to analytics, they are just not compensated by them or driven by them.

Sam Stites, Editor-in-chief of The Daily Emerald, the University of Oregon's student newspaper said:

New writers need to hone their reporting and editing skills before diving into analytics.

At Salon, "The editors allow writers to see metrics if they ask, but they don't actively share traffic numbers other than a surface-level weekly report on popular stories."

As, Salon political writer Brian Beutler said:

I don't think that it's super important for individual writers to be thinking about that on a micro level, moment to moment," said Salon editor-in-chief David Daley. "We want them doing their jobs and thinking about stories.

Even at the LA Times where journalists have access to analytics, the notion of actually using it is held at arm's length. As Jimmy Orr, digital managing editor said:

I think we have to embrace metrics and they can tell us a lot about the reader and what they're looking for, but it's not the end all, be all, at all.

And it's not just a problem for newspapers and magazines. Brands too often have the same attitude. For instance, the fashion company I used to work for had a policy that prevented marketing from measuring sales. As such, we had no way to measure real-world ROI.

Imagine if any of these people walked into the pit of a race team with this type of attitude towards data.

And again, race teams are winning not just because they use data, but also because they understand it. They have spent an enormous amount of time testing and experimenting with their data. They know exactly what each data point influences, and what real-world impact it has on their overall performance.

For instance, if they see that the tire-pressure is dropping in one of their tires, they can use this data to determine exactly how big an impact it will have, and use that to plan ahead for when to bring the car back into the pit. Can they wait for the next planned pitstop (which would be preferable), or will it degrade their performance so quickly that they have to do it now? And if they do, they also look at the metric for fuel consumption and many other factors to determine if it would be beneficial to look at that too.

So the challenge for the media industry is not just to look at the data as an integrated and vital part of running your business. But also to understand it.

And when you start doing that, you realize that 99% of the analytics you have today is completely and totally pointless. You are measuring the wrong things, because you don't understand how the data impacts your business.

It's time to fix that!

BTW: If you want to see what it takes to win at Le Mans, watch the brilliant documentary from Audi: 'Truth in 24 II', a sequel to the equally excellent 'Truth in 24'

 

More from Baekdal